DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-12 are pending.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3-7, and 10-12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al. (US 2016/0379107).
With respect to claim 1, Li discloses: receiving, by at least one IoT device, the user input from a user of the at least one IoT device to execute at least one task associated with the at least one IoT device in the IoT environment ([0119], [0120]; “please answer at once” corresponds to Applicant’s “user input”, the act of answering the call and providing the video communication is interpreted as the “task”);
acquiring, by the at least one IoT device, multimodal data of the IoT environment based on the user input ([0120], [0141]; the camera identifies the direction of the speaker and turns the camera to the direction of the speaker, the image and sound of the speaker corresponds to Applicant’s “multimodal data”);
predicting, by the at least one IoT device, a task execution intensity for the at least one task associated with the at least one IoT device based on the user input and the multimodal data ([0120]; “double camera switch algorithm ensures picture of camera is stable and does not shake” corresponds to Applicant’s “task execution intensity”. The claim does not disclose how the “predicting” is accomplished. The broadest reasonable interpretation of “predicting” is anticipating that the camera will shake and employ the algorithm to ensure the camera is stable and does not shake); and
executing, by the at least one IoT device, the at least one task associated with the at least one IoT device with the predicted task execution intensity ([0120], [0121]).
With respect to claim 3, Li discloses: wherein acquiring, by the at least one IoT device, the multimodal data of the IoT environment comprises: determining, by the at least one IoT device, a multimodal context of the IoT environment relevant to the at least one task associated with the at least one IoT device based on the received user input; and acquiring, by the at least one IoT device, the multimodal data of the IoT environment corresponding to the determined multimodal context ([0119]-[0121]; camera activates in response to user input).
With respect to claim 4, Li discloses: wherein the multimodal context comprises at least one of a context of the user, a context of the at least one IoT device and an ambient context, and wherein the context of the user context is determined from one or more derived input from the multimodal data, pertaining to an user activity, and state of connected IoT devices, wherein the ambient context is determined from one or more derived inputs from IoT device data, non-speech scene detection, sensory output, and an external parameter ([0119]-[0121]).
With respect to claim 5, Li discloses: wherein the multimodal data comprises at least one of: a gesture of the user, Ultra-wideband (UWB) position of the at least one IoT device, data associated with the at least one IoT device, at least one sensor input, feed associated with an imaging device, voice assistant information, and non-speech information ([0120]; camera corresponds to “imaging device”).
With respect to claim 6, Li discloses: wherein the task execution intensity comprises at least one of: a functional mode of the at least one IoT device, a position of the at least one IoT device, a movement of the at least one IoT device, and a control function of the at least one IoT device ([0120]).
With respect to claim 7, Li discloses: wherein the multimodal data is acquired at least by: receiving at least one of the user input, a gesture of the user, Ultra-wideband (UWB) position of the at least one IoT device, data associated with the at least one IoT device, at least one sensor input, feed associated with an imaging device, voice assistant information, and non-speech information to generate the multimodal data; and converting and normalizing the generated multimodal data ([0119], [0120], [0140]).
With respect to claim 10, Li discloses: wherein the task execution intensity is determined using at least one of a machine learning (ML) based technique, a Random forest technique, a clustering based technique and a decision tree based classifier, wherein the task execution intensity is determined based on at least one of capability of the at least one IoT device, a state of the at least one IoT device, and an execution control data associated with the at least one IoT device ([0119]-[0121], the decisions made by the IoT device corresponds to “decision tree”).
With respect to claims 11 and 12, they recite similar limitations as claim 1 and is therefore rejected under the same citations and rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 2016/0379107) in view of Choi et al. (US 2019/0140856).
With respect to claim 2, Li does not specifically disclose: monitoring, by the at least one IoT device, the task execution intensity for the at least one task as a feedback over a period of time; and executing, by the at least one IoT device, the at least one task associated with the at least one IoT device based on the feedback.
However, Choi discloses: monitoring, by the at least one IoT device, the task execution intensity for the at least one task as feedback over a period of time; and executing, by the at least one IoT device, the at least one task associated with the at least one IoT device based on the feedback ([0127]- [0130]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to incorporate feedback as taught by Choi to enable Li’s IoT device to leverage active learning, transfer learning, and reinforcement learning to further enhance machine learning models as taught by Choi, thereby improving Li’s algorithm to reduce camera shaking or provide alerts for emergencies.
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 2016/0379107) in view of Mueen et al. (US 10853372)
With respect to claim 9, Li does not specifically disclose: wherein the multimodal data is updated over a period of time, using a data driven model, based on at least one of the user behavior, an user usage pattern and the at least one IoT device, wherein the multimodal data is processed using a map reduction technique.
However, Mueen discloses: wherein the multimodal data is updated over a period of time, using a data driven model, based on at least one of the user behavior, an user usage pattern and the at least one IoT device, wherein the multimodal data is processed using a map reduction technique (col. 16, lines 36-54).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to incorporate the teachings of Mueen’s map reduce technique to allow processing large datasets by dividing them into smaller, manageable tasks that can be processed in parallel and significantly improve performance by distributing the tasks to multiple worker/threads/processors.
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Li discloses receiving, by at least one IoT device, the user input from a user of the at least one IoT device to execute at least one task associated with the at least one IoT device in the IoT environment ([0119], [0120]; “please answer at once” corresponds to Applicant’s “user input”, the act of answering the call and providing the video communication is interpreted as the “task”); acquiring, by the at least one IoT device, multimodal data of the IoT environment based on the user input ([0120], [0141]; the camera identifies the direction of the speaker and turns the camera to the direction of the speaker, the image and sound of the speaker corresponds to Applicant’s “multimodal data”); predicting, by the at least one IoT device, a task execution intensity for the at least one task associated with the at least one IoT device based on the user input and the multimodal data ([0120]; “double camera switch algorithm ensures picture of camera is stable and does not shake” corresponds to Applicant’s “task execution intensity”. The claim does not disclose how the “predicting” is accomplished. The broadest reasonable interpretation of “predicting” is anticipating that the camera will shake and employ the algorithm to ensure the camera is stable and does not shake); and executing, by the at least one IoT device, the at least one task associated with the at least one IoT device with the predicted task execution intensity ([0120], [0121]).
However, in combination with the above limitations, the prior art does not disclose mapping a wearable device and a sensor data to obtain current environmental state of the user and at least one IoT device and obtain positional information, current operation state, content and operation intensity status from an imaging device and a non-speech feed and then acquiring multimodal data based on current state of the user IoT device, obtained position, operational status and intensity status.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Jarosz et al. (US 12531057): A system and method of responding to a vocal utterance may include capturing and converting the utterance to word(s) using a language processing method, such as natural language processing. The context of the utterance and of the system, which may include multimodal inputs, may be used to determine the meaning and intent of the words.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WISSAM RASHID whose telephone number is (571)270-3758. The examiner can normally be reached Monday-Friday 8:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aimee Li can be reached at (571)272-4169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WISSAM RASHID/Primary Examiner, Art Unit 2195
Prosecution Insights