DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-13 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13 are rejected under 35 U.S.C. 102(a)(1) as anticipated by or, in the alternative, under 35 U.S.C. 103 as obvious over Bouchabou (“A Smart Home Digital Twin to Support the Recognition of Activities of Daily Living,” Published September 1, 2023 – Provided on IDS).
As to claim 1, Bouchabou discloses a method for determining a human activity in an environment comprising at least one usage sensor for at least one device of the environment (Bouchabou at Page 9, item 4, IoT sensors; Page 10, Section 3.2.1, smart sensor array; Fig. 2), the method being implemented by at least one processing circuit comprising an artificial intelligence (Bouchabou at Page 4, in particular, 2.1 Machine Learning Algorithms for Activity Recognition Based on Smart Home Iot Data) and comprising:
learning by the artificial intelligence and an inference by the artificial intelligence in order to determine the human activity in said environment (Bouchabou at Page 14, Section 4, last paragraph, in particular; Page 18, Section 4.3.3) and
for the learning by the artificial intelligence: generating artificial intelligence training data, based on a simulation of an activity of at least one autonomous agent operating within a simulation of the environment generated from a digital twin of the real environment and using at least one digital replica of said device (Bouchabou at Fig. 4; Pages 13-14, 3.2.3 The Synthetic Dataset disclose “To generate the synthetic dataset, we first recreated our living lab within the VirtualSmartHome simulator. The objective was to create a digital twin of the physical space, ensuring that each object and room was accurately represented with corresponding dimensions. A visual comparison between the real living lab and its virtual representation is shown in Figure 4…. The synthetic dataset was generated by replicating the recorded scenarios from the living lab within the simulator…. We will present our results in three sections to validate our digital twin approach and draw conclusions regarding activity recognition using synthetic data”).
As to claim 2, Bouchabou discloses the method according to claim 1, wherein the inference by the artificial intelligence is carried out based on data captured during the use of said at least one device by at least one physical user, while having knowledge of the training data acquired from simulated uses of the digital replica of the device by said at least one autonomous agent (Abstract discloses “we demonstrate that an activity recognition algorithm trained on the data generated by the VirtualSmartHome simulator can be successfully validated using real-life field data.” Section 3.1 discloses “Virtual Home is a multi-agent platform designed to simulate activities in a house or apartment. It utilizes humanoid avatars that can interact with their environment and perform activities based on high-level instructions. The simulator incorporates a knowledge base that enables the creation of videos depicting human activities, as well as training agents to perform complex tasks.” Section 3.2.2. The Ground Truth Dataset, Section 3.2.3. The Synthetic Dataset discloses “The synthetic dataset was generated by replicating the recorded scenarios from the living lab within the simulator. We scripted an avatar to mimic each action performed by our volunteers. For example, if a volunteer followed these steps for the activity “cooking”: entering the kitchen, washing hands, opening the fridge, closing the fridge, turning on the oven, etc., we scripted the avatars in the VirtualSmartHome simulator to simulate each of these actions. We created one script for each occurrence of an activity performed by our volunteers in all three scenarios, allowing us to obtain a database of scripts that we can reuse later for other environment configurations.” Page 21 discloses “In more detail, three models were trained, one for each subject using his own synthetic and his own real data.”).
As to claim 3, Bouchabou discloses the method according to claim 2, wherein the at least one autonomous agent used for generating the training data is selected based on a profile of the at least one physical user (Section 3.2.2.).
As to claim 4, Bouchabou discloses the method according to claim 1, comprising initializing the digital twin of the real environment prior to generating training data for the artificial intelligence, said digital twin being configured to reproduce a state and a behavior of the real environment by simulating the environment of said real environment, comprising the digital replica of said at least one device in the real environment (Bouchabou at Fig. 4; Page 9).
As to claim 5, Bouchabou discloses the method according to claim 4, wherein the digital twin is generated from data of the real environment, including at least: a type of business premises or residential premises, a surface area of said premises, functional spaces in said surface area, and types of functions of said spaces, a state of said premises (Bouchabou at Figs. 2-4; Page 9).
As to claim 6, Bouchabou discloses the method according to claim 1, wherein said at least one device in the real environment comprises at least: a sensor for detecting an opening and/or closing of a water inlet; a sensor for a connected power outlet; a presence sensor; a pressure sensor; a sensor for detecting the turning on and/or off of a light; a motion sensor for a door, blind, shutter, and/or window; a thermostatic sensor; a usage sensor for a household appliance; and/or a usage sensor for a connected terminal (Bouchabou at Figs 2, 4; Page 9).
As to claim 7, Bouchabou discloses the method according to claim 1, wherein said at least one sensor is connected to a local area network linked to a gateway connected to a server via a wide area network (Bouchabou at Abstract; Page 9 IoT sensor are necessarily connected the Internet).
As to claim 8, Bouchabou discloses the method according to claim 1, further comprising monitoring an efficiency of the artificial intelligence, wherein the inference by the artificial intelligence is implemented on the basis of data captured during the simulation of the activity of the at least one autonomous agent operating within the simulation of the environment generated from the digital twin of the real environment, while having knowledge of the training data acquired from simulated uses of the digital replica of the device by the at least one autonomous agent, wherein a comparison of the inference produced by the artificial intelligence and the simulation of the activity of the at least one autonomous agent is evaluated, and if the comparison is greater than a predefined threshold, said training data is corrected relative to said data captured during the simulation of the activity of the at least one autonomous agent (Bouchabou at Section 4.4 discloses “The activity recognition algorithm performed similarly on both synthetic and real data, indicating that training the algorithm solely on synthetic data can effectively recognize activities in real-world scenarios. Moreover, when the entire set of generated synthetic data was utilized, the algorithm’s performance improved for each subject. This improvement can be attributed to the increased variability and examples provided by the additional synthetic data, allowing the algorithm to better generalize and capture the behavior of sensors during different activities”).
As to claim 9, Bouchabou discloses the method according to claim 8, comprising generating at least one new autonomous agent, depending on said comparison exceeding the predefined threshold, in order to generate new training data for the artificial intelligence (Bouchabou at page 24 discloses “However, we acknowledge the need for further in-depth discussion and analysis to gain deeper insights from the results. In future work, we intend to explore the limitations of our study, specifically focusing on the impact of data collection in a lab environment versus a real-world home and the significance of dataset size. Understanding these aspects is critical for assessing the generalizability and practical applicability of our proposed approach. To achieve this, we plan to expand the experiment by generating more synthetic data from additional volunteers’ activities. Additionally, we aim to extend the evaluation to include a larger number of real smart houses, allowing for a more comprehensive assessment of our approach’s performance across diverse environments. Furthermore, we will explore the possibility of integrating scenarios with multiple agents to enrich datasets with more complex situations.”).
As to claim 10, Bouchabou discloses the method according to claim 1, comprising an increase in a speed of operating of the at least one autonomous agent within said simulation of the environment generated from the digital twin, compared to an actual operating of at least one physical user within the real environment, in order to accelerate the simulation of activity by the at least one autonomous agent and of the generation of training data (Bouchabou at page 9, item 2, Simulation time acceleration;” page 25 discloses “In our future work, we aim to investigate training the algorithm from scratch in a house without replicating the labeled activities of the final resident. Instead, we will solely utilize the activity scripts provided by our volunteers, enabling a more realistic and autonomous training process. By addressing these aspects in our future work, we aim to further validate and enhance the effectiveness of our approach for generating synthetic data and training HAR algorithms in smart homes”).
As to claim 11, Bouchabou discloses the method according to claim 1, wherein each autonomous agent operates in said simulation of the environment according to a decision made by said autonomous agent based on a model of needs of said autonomous agent (Bouchabou at Page 25 discloses ““In our future work, we aim to investigate training the algorithm from scratch in a house without replicating the labeled activities of the final resident. Instead, we will solely utilize the activity scripts provided by our volunteers, enabling a more realistic and autonomous training process. By addressing these aspects in our future work, we aim to further validate and enhance the effectiveness of our approach for generating synthetic data and training HAR algorithms in smart homes”)”).1
As to claim 12, Bouchabou discloses a non-transitory computer readable medium comprising a computer program stored thereon comprising instructions for implementing a method (Bouchabou at Fig. 1; Section 3, Virtual Smart Home: The Simulator) for determining a human activity in an environment comprising at least one usage sensor for at least one device of the environment (Bouchabou at Page 9, item 4, IoT sensors; Page 10, Section 3.2.1, smart sensor array; Fig. 2), when the instructions are executed by a processing circuit (Bouchabou at Page 4, in particular, 2.1 Machine Learning Algorithms for Activity Recognition Based on Smart Home Iot Data), the method comprising:
learning by an artificial intelligence and an inference by the artificial intelligence in order to determine the human activity in said environment (Bouchabou at Page 14, Section 4, last paragraph, in particular; Page 18, Section 4.3.3); and for
the learning by the artificial intelligence: generating artificial intelligence training data, based on a simulation of an activity of at least one autonomous agent operating within a simulation of the environment generated from a digital twin of the real environment and using at least one digital replica of said device (Bouchabou at Fig. 4; Pages 13-14, 3.2.3 The Synthetic Dataset disclose “To generate the synthetic dataset, we first recreated our living lab within the VirtualSmartHome simulator. The objective was to create a digital twin of the physical space, ensuring that each object and room was accurately represented with corresponding dimensions. A visual comparison between the real living lab and its virtual representation is shown in Figure 4…. The synthetic dataset was generated by replicating the recorded scenarios from the living lab within the simulator…. We will present our results in three sections to validate our digital twin approach and draw conclusions regarding activity recognition using synthetic data”).
As to claim 13, Bouchabou discloses a device comprising: a processing circuit comprising an artificial intelligent configured to determine a human activity in an environment comprising at least one usage sensor for at least one device of the environment (Bouchabou at Page 4, in particular, 2.1 Machine Learning Algorithms for Activity Recognition Based on Smart Home Iot Data) by:
learning by the artificial intelligence and an inference by the artificial intelligence in order to determine the human activity in said environment (Bouchabou at Page 14, Section 4, last paragraph, in particular; Page 18, Section 4.3.3); and
for the learning by the artificial intelligence: generating artificial intelligence training data, based on a simulation of an activity of at least one autonomous agent operating within a simulation of the environment generated from a digital twin of the real environment and using at least one digital replica of said device (Bouchabou at Fig. 4; Pages 13-14, 3.2.3 The Synthetic Dataset disclose “To generate the synthetic dataset, we first recreated our living lab within the VirtualSmartHome simulator. The objective was to create a digital twin of the physical space, ensuring that each object and room was accurately represented with corresponding dimensions. A visual comparison between the real living lab and its virtual representation is shown in Figure 4…. The synthetic dataset was generated by replicating the recorded scenarios from the living lab within the simulator…. We will present our results in three sections to validate our digital twin approach and draw conclusions regarding activity recognition using synthetic data”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Karri (US 2023/098596 A1, Published March 30, 2023) is made of record for its relevance to claims 1, 12, 13 by its disclosure of the following in Fig. 4:
PNG
media_image1.png
581
667
media_image1.png
Greyscale
Mattis (US 2022/0379169 A1, Published December 1, 2022) is made or record for its relevance to claims 1, 12, and 13 by its disclosure in Fig. 3:
PNG
media_image2.png
672
497
media_image2.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sanjiv D Patel whose telephone number is (571)270-5731. The examiner can normally be reached Monday - Friday, 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Sanjiv D. Patel/Primary Examiner, Art Unit 2625
07/20/2026
1 See also Gramoli, “Generating and Evaluating Data of Daily Activities with an Autonomous Agent in a Virtual Smart Home,” May 22, 2024 – provided on IDS.
Prosecution Insights