DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Great Britain on 09/29/2023. It is noted, however, that applicant has not filed a certified copy of the GB2315037.8 application as required by 37 CFR 1.55.
Claim Objections
Claims 2,12,17 and 20 are objected to because of the following informalities:
In line 4 of claim 2, the phrase, “…based on the determine deviation…” has a grammatical error. Suggested correction, replace “determine” with “determined”.
In line 4 of claim 12, the phrase, “…based on the determine deviation…” has a grammatical error. Suggested correction, replace “determine” with “determined”.
In line 5 of claim 17, the phrase, “…based on the determine deviation…” has a grammatical error. Suggested correction, replace “determine” with “determined”.
Also in claim 17, the phrase, “The non-transitory computer-readable storage medium of claim 16, further comprising: determining, based…”, is incorrect because the non-transitory computer-readable storage medium is not executing the functional limitation of determining rather the method performed when executed by the device executing the functional limitation of determining. Suggested correction, replace the phrase, “further comprising:” with “wherein the method further comprises:”.
Similarly, in claim 20, the phrase, “The non-transitory computer-readable storage medium of claim 16, further comprising: determining, based…”, is incorrect because the non-transitory computer-readable storage medium is not executing the functional limitation of determining rather the method performed when executed by the device executing the functional limitation of determining. Suggested correction, replace the phrase, “further comprising:” with “wherein the method further comprises:”.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Claim 1 recites the generic placeholder, “device” followed by functional limitations without reciting structure to perform the functional limitations. For support of structure, examiner looked into [0110]-[0119] of the specification which recites the device as client device having various components such as memory, transmitter and others executing the functional limitations.
Claim 16 recites the generic placeholder, “device” followed by functional limitations without reciting structure to perform the functional limitations. For support of structure, examiner looked into [0110]-[0119] of the specification which recites the device as client device having various components such as memory, transmitter and others executing the functional limitations.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Shive et al. (US 20190349213 A1) in view of Gavin et al. (US 20240401833 A1).
Regarding claim 1, Shive et al. teaches, a method comprising:
identifying, by a device (smart home system having controlling device, [0005] and [0055]), an event occurring at a location (user in kitchen, [0006]), the event corresponding to activity performed by a user at the location (user starts cooking, [0006]), the location configured with a control system for managing activities at the location (In particular embodiments, the smart home system 100 may detect the user's activities automatically based on the sensor data without receiving any commands from the user…the smart home system 100 may detect some activities of the user using the data from a motion sensor. The smart home system 100 may interpret the user's activities using the multiple layers of knowledge which indicates that the user usually starts to cook for dinner about 5:00 pm and the smart home system 100 may detect that the current time is 5:05 pm and the user is in the kitchen. The smart home system 100 may determine that the user is about to start cooking and may turn on the light in the kitchen based on the user preference of kitchen light for cooking1…”, [0006], see also [0041] and [0159]);
analyzing, by the device, the event (hit the light switch in the kitchen, [0159]), and determining, based on the analysis, attributes related to the event, the attributes comprising information related to the activity (“…if the user is cooking in the kitchen, the user may hit a light switch to turn on the task lighting. The inference subsystem2 may use a combination of contextual information to determine the user's most likely intent with hitting the light switch, and the most likely activity…”, [0159]);
identifying, by the device from a database (the controlling device of smart home system accessing the database to retrieve or save information related to user activity and other sensor data, [0157]), a pattern of activity for the user at the location, the pattern corresponding to previous actions by the user at a time at the location, the pattern comprising information indicating electronic controls of the control system based on the previous actions, (“…In particular embodiment, the inference process may enable the smart home system to take action that is most appropriate for the given situation. For example, instead of turning the kitchen task
lights on to their last known setting3 or to a static default setting, the system may turn them on to an appropriate color temperature and brightness level for cooking, which may be different than the ideal settings when the same user hits the same light switch late at night or when the user is passing through the kitchen on their way to the bathroom. In particular embodiment, the inference process may enable the smart home system to take action that the user necessarily thinks of taking themselves…”, [0160], see also [0127]);
the determined attributes of the event and the identified pattern of activity (“…The smart home system 100 may interpret the user's activities using the multiple layers of knowledge which indicates that the user usually starts to cook for dinner about 5:00 pm and the smart home system 100 may detect that the current time is 5:05 pm and the user is in the kitchen4. The smart home system 100 may determine that the user is about to start cooking and may turn on the light in the kitchen based on the user preference of kitchen light for cooking…’, [0006], see also [0240]); and
automatically executing, by the device, the electronic controls of the control system based on the feedback and the identified pattern of activity (“…In particular embodiments, the system may use a feedback loop utilizing sensor data and user input to refine its learned understanding of how various device settings affect a
given area, and/or how certain device attributes affect real-world outcomes, and/or user preferences, to be used for later synthesis of device settings. In particular embodiments, the system may use feedback (e.g., sensor and user input), after a synthesized device state is set, to determine if the intent and/or activity was correctly inferred…”, [0181], that is based on user feedback and user pattern identification, control the lights in the house as taught in [0182], see also [0231]).
Shive et al. does not teach the details of analyzing, by the device, via a large language model (LLM), the determined attributes of the event and the identified pattern of activity; generating, by the device, an LLM prompt based on the analysis, the LLM prompt being automatically output at the location in response to the identification of the event; receiving by the device, feedback from the user to the LLM prompt. However Shive et al. teaches to analyze the event and the identified pattern, automatically output commands in proper location to execute instructions, and receive feedback from the user as taught in [0006], [0116] and [0160]. However, it is not clear exactly what type of AI model is used by Strive et al.
Gavin et al. teaches, analyzing, by the device, via a large language model (LLM) (analyzing building equipment data using large language model, [0009] and [0095]);
generating, by the device, an LLM prompt based on the analysis, the LLM prompt being automatically output at the location in response to the identification of the event (“…the optimization engine 120 may receive a that asks, “why is room A always hot in the morning?5” and the optimization engine 120 may generate an optimization improvement for the building based on the prompt. The optimization engine 120 may retrieve or access building specs, building layouts, room layouts, geographical data, or BIM models to determine a location of the room A with the building. The optimization engine 120 may determine that room A includes multiple windows that receive several hours of sunlight each day. The optimization engine 120 may generate or update a control schedule for blinds associated with the windows to reduce an amount of heat 6that is produced as a result of the sunlight…”, [0095], that is using the large language model, when prompted why the room A is hot, the LLM analyzed several data related to room A and identified updated control schedule for controlling the blinds in room A, see also [0181]);
receiving by the device, feedback from the user to the LLM prompt (receiving user feedback related to the controls performed by the LLM (second model as taught in [0102] and [0179]).
Shive et al. and Gavin et al. are analogous art because they are from the same field of endeavor that is home automation systems.
Therefore it would have been obvious before the effective filing date of the claimed invention to a person of ordinary skill in the art to modify the method of identifying an event, analyzing the event and execution device control based on identified and analyzed event as taught by Shive et al. by applying the known technique of using large language model to perform identification and analyzation as taught by Gavin et al. to yield predictable results of automatically controlling the control system based on event identification and analysis.
Shive et al. teach:
[0182] In particular embodiments, the system may have a rewriting process during the intent inference. The rewriting process by the system may result in different commands even for the same user inputs. As an example and not by way of limitation, the system may receive the command of turn off light” and may determine different commands based on notion of light, scope to whole home and other factors using the rewriting process. As another example and not by way of limitation, when the system receives user command of “turn off light”, the system may determine that the command is
received at bed time and user may mean to turn off the light in the whole home to go to bed. Although the command has no explicit location specified, the system may use contextual information to re-infer the user's intent. As another example and not by way of limitation, user may be in the living room and tell the system to turn of the light. The system may determine that the user does not meant to turn off all light because the user is in the living room and may need light to go to bedroom. The system may re-interpret that command from turn off light to something like “start to shut down the home for
preparing for sleep but keep some dim light along the way from living room to bedroom7”.
Gavin et al. teach:
[0102] The feedback repository 124 can include feedback received from users regarding output presented by the applications 120. For example, for at least a subset of outputs presented by the applications 120, the applications 120 can present one or more user input elements for receiving feedback regarding the outputs. The user input elements can include, for example, indications of binary feedback regarding the outputs (e.g., good/bad feedback; feedback indicating the outputs do ordo not meet the user's criteria, such as criteria regarding technical accuracy or precision); indications of multiple levels of feedback (e.g., scoring the outputs on a predetermined scale, such as a 1-5 scale or 1-10 scale); freeform feedback (e.g., text or audio feedback); or various combinations thereof.
[0179] In some implementations, the AI model may take as input user feedback regarding one or more goals, or a balance of one or more goals. For example in some implementations, a user interface maybe provided allowing the user to identify an absolute or relative amount of emphasis to put on one or more goals, or prioritize between a plurality of goals. In some such implementations, the interface may include a slider, multi-dimensional coordinate system, alphanumeric weighting interface, or other interface allowing the user to indicate a priority between multiple goals, such as between cost, energy usage, carbon emissions or other sustainability metrics, occupant comfort or health metrics, air quality, and/or other factors. In some implementations, the user may be provided with a conversational chat interface through which the user, via text, audio, visual, etc. input can indicate priorities or other input guidance for the model in determining the output parameters. For example, in some implementations, the input may be an unstructured natural language input, such as input that does not conform to a predetermined query ontology or input that conforms to a plurality of different query ontologies. In some such implementations, the model may be a generative AI such as a generative (LLM) (e.g., a generative AI chat model)….
Regarding claim 2, combination of Shive et al. and Gavin et al. teach the method of claim 1. In addition, Strive et al. teaches, determining, based on analysis of the feedback, that the activity of the event performed by the user deviates from the pattern of activity (when the system determine the user is not home by 8:00pm where the user is usually home by 7:00pm, the system asks the user when the user is coming home, [0240] and [0242]); and
modifying the electronic controls based on the determine deviation (based on detected deviation, the anomaly detection system automatically informs the user, and controls the lighting around the house, [0240] and [0242]), wherein the automatically executed electronic controls are the modified electronic controls (based on users’ answer to the systems question/prompt8, the system determines to control the alarm system, lights and other component of the house, [0240] and [0242]).
Regarding claim 3, combination of Shive et al. and Gavin et al. teach the method of claim 1. In addition, Strive et al. teaches, wherein the attributes further comprise information related to a context of the activity (activity pattern of the user, [0265]), wherein the context is based on information related at least to an environment in or around the location (contextual information of the environment, [0264]), personal data of the user derived from third party applications (routine of the user and characteristics of the user activity and local weather data of the house from outside9 [0122] and [0265]) and activities of other users in or around the location (The smart home system monitors a space where multiple users present in a certain location such as bedroom. Also when an intruder10 around the house is detected, the user is informed, also when a guest is at the front door, the user is informed, [0148] and [0240]).
Regarding claim 4, combination of Shive et al. and Gavin et al. teach the method of claim 1. In addition, Gavin et al. teaches, wherein the LLM prompt is configured in a format as at least one of text, audio, video and an electronic message (the prompt from the user related to the LLM is received in the form of text, audio, speech, image or video format, [0144] and [0166]), wherein the output of the LLM prompt is performed in a manner that corresponds to the format (“…At 825, the completion is presented via the application session11. For example, the completion can be presented as any of text, speech, audio, image, and/or video data to represent the completion, such as to provide an answer to a query represented by the regarding an item of equipment or building management system…”, [0171], that is an output by the LLM to user prompt is provided in the form of text, speech or other format as received from the user).
Regarding claim 5, combination of Shive et al. and Gavin et al. teach the method of claim 4. In addition, Gavin et al. teaches, wherein the feedback is provided in the format of the LLM prompt (the feedback from the user related to LLM output (prompt) can be received in various formats such as text, speech and others, [0140] and [0144]).
Regarding claim 6, combination of Shive et al. and Gavin et al. teach the method of claim 1. In addition, Shive et al. teaches, further comprising: monitoring the location according to a criteria and based on a type of the control system (the smart home system use a profile of activity (criteria), real time sensor data and other meta data to determine the likely activities in a given place, [0231]);
and
collecting sensor data from at least one sensor at the location based on the monitoring, wherein the identification of the event is based on the collection of the sensor data (when the motion sensor in the kitchen detects a user in the kitchen, the smart home system automatically may detect the cooking activity12 and automatically light up the kitchen appropriately, [0231] and [0232]).
Regarding claim 7, combination of Shive et al. and Gavin et al. teach the method of claim 5. In addition, Shive et al. teaches, further comprising:
identifying a set of devices associated with the location (identify the cooking apparatus and kitchen lights in the kitchen, [0056] and [0231]);
collecting data from each of the set of devices (collecting real time sensor data and metadata in the kitchen, [0231]);
analyzing, via an application, the collected data (the smart home system analyzes the collected sensor data and metadata to determine likely activities on the kitchen such as user starts to cook, user wanting to adjust lighting settings when turning knobs, [0160] and [0231]);
determining, via the application, a set of patterns of activity for the user (the smart home system knows the user usually starts cooking at 5:00pm, the smart home system combine the detected activities or habits of the users with knowledge graph and use the combination to interpret current activities and determine ideal conditions of the home for these activities such as cooking light settings, [0231], see also [0241] and [0148]); and
storing, in the database, the set of patterns of activity (the smart home system stores the detected activities as activities profiled in the database, [0232]), wherein the identified pattern of activity is retrieved from the stored set of patterns of activity (the smart home system can access the activity profiles stored in the database to determine or infer user activities, [0232]).
Regarding claim 8, combination of Shive et al. and Gavin et al. teach the method of claim 7. In addition, Shive et al. teaches, wherein the set of devices correspond to a type of sensor related to the type of the control system (in the kitchen the devices include motion sensor, cooking apparatus, and the kitchen lights, [0056] and [0231]).
Regarding claim 9, combination of Shive et al. and Gavin et al. teach the method of claim 1. In addition, Shive et al. teaches, wherein the control system is a climate system configured to control a climate at the location (smart home system controlling thermostat in the house based on mode of operation of the house, [0056] and [0242]), wherein the device is a thermostat (thermostat controlled by the smart home system to control temperature, [0056] and [0242]).
Regarding claim 10, combination of Shive et al. and Gavin et al. teach the method of claim 1. In addition, Shive et al. teaches, wherein the control system is a security system configured to provide anti-intrusion activities at the location (anomaly detection system at the door of the smart home, [0142]), wherein the device is a security control panel (the dialog system and the security cameras of the anomaly detection system detects intruder and informs the user about the intruder, [0082] and [0142]).
Regarding claim 11, combination of Shive et al. and Gavin et al. teach the claimed method having a device for identifying, analyzing an event and automatically executing electronic controls in response to the event. Therefore together they teach the device performing the functional limitations for identifying, analyzing an event and automatically executing electronic controls in response to the event as taught in claimed method of claim 1. Claim 11 has an additional limitation, which is taught by Shive, device comprising a processor (smart home system having controlling device have processor, [0061]).
Regarding claims 12-15, combination of Shive et al. and Gavin et al. teach the claimed method having a device for identifying, analyzing an event and automatically executing electronic controls in response to the event. Therefore, together they teach the device performing the functional limitations for identifying, analyzing an event and automatically executing electronic controls in response to the event as taught in claimed method of claims 2,3,4 and 6.
Regarding claim 16, combination of Shive et al. and Gavin et al. teach the claimed method having a device for identifying, analyzing an event and automatically executing electronic controls in response to the event. Therefore, together they teach the device performing the functional limitations for identifying, analyzing an event and automatically executing electronic controls in response to the event as taught in claimed method of claim 1. Claim 16 has an additional limitation, which is taught by Shive, a non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a device (computing system having controlling device as taught in [0061] has a computer-readable non-transitory medium including one or more computer components for executing instructions, [0271] and [0279]).
Regarding claims 17-20, combination of Shive et al. and Gavin et al. teach the claimed method having a device for identifying, analyzing an event and automatically executing electronic controls in response to the event. Therefore, together they teach the method performing the functional limitations for identifying, analyzing an event and automatically executing electronic controls in response to the event as taught in claimed method of claims 2,3,4 and 6.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Spero (US 11651258 B2) teaches a device providing localized delivery of services such as lighting, heating, cooling and others to the user based on sensor data and learned user data.
Wilson (US 20240420214 A1) teaches a home automation system implementing large language model to determine home improvements, predictive maintenance of the devices within a home and to perform corrective actions to mitigate abnormal activities within the home.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANZUMAN SHARMIN whose telephone number is (571)272-7365. The examiner can normally be reached M and Th 7:00am - 3:00pm and Tue 8:00am-12:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KAMINI SHAH can be reached at (571)272-2279. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANZUMAN SHARMIN/ Examiner, Art Unit 2115
/KAMINI S SHAH/ Supervisory Patent Examiner, Art Unit 2115
1 The smart home system managing lights in the kitchen when determined user is in the kitchen and is about to start cooking.
2 Part of smart home system analyzing user’s action such as hitting the light switch (event) to determine appropriate lighting control.
3 Pattern corresponding to previous actions such as last known setting of lights in the kitchen by the user when the user was cooking. The system knows when the user is cooking in the kitchen vs when the user is passing by the kitchen based on learned user activity/pattern of behavior and time of the day as taught in [0148] and [0201].
4 Based on user identified pattern that is user usually starts cooking at 5:00pm and user is now in the kitchen (event), the system automatically adjusts the kitchen light settings.
5 LLM prompt from the user.
6 Automatically outputting a response in the location room A based on the received LLM prompt from the user.
7 Based on time of the day, location and user behavior, the system can infer user’s intent and determine appropriate light settings and control the lights accordingly.
8 Analysis to the feedback.
9 Third party
10 Users in or around the location
11 Output presented to the user.
12 Identifying event based on the collection of sensor data.
Prosecution Insights