DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed 3/3/23 fails to comply with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609 because application number not included within IDS. It has been placed in the application file, but the information referred to therein has not been considered as to the merits. Applicant is advised that the date of any re-submission of any item of information contained in this information disclosure statement or the submission of any missing element(s) will be the date of submission for purposes of determining compliance with the requirements based on the time of filing the statement, including all certification requirements for statements under 37 CFR 1.97(e). See MPEP § 609.05(a).
Drawings
The drawings filed 3/3/23 have been reviewed and accepted.
Response to Amendment
The amendment filed 3/3/23 has been entered.
Claims 1-5 and 7 remain pending within the application.
Claim Objections
Claim 7 is objected to because of the following informalities:
“constituting a decision tree that branches in the order of extraction, constituted decision tree;,” should be “constituting a decision tree that branches in the order of extraction[[,]] .
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Graefe United States Patent Application Publication US 2019/0222652 in view of Jaggannath et al “Machine Learning for Wireless Communications in the Internet of Things: A Comprehensive Survey” Ad Hoc Networks 93 (7/18/19).
Regarding claim 1, Graefe discloses a device control value generation device, configured to generate device control values of a plurality of control target devices, the device control value generation device comprising a processor configured to perform operations (Graefe, para [0132 and 133], fig 10, example computer platform 1000 includes processor 1002, memory 1060. Computer connected to sensors 1021 and actuators 1022) comprising:
acquiring data from each IoT device (Graefe, para [0052], collects data from sensors 62), determining an external factor according to a type of the IoT device (Graefe, para [0053], received sensor data represents external factors. Type of sensor would modify the type of sensor data), and determining to which division range obtained by dividing an upper limit value and a lower limit value of the determined external factor into a predetermined range the acquired data belongs (Graefe, para [0030 and 55], sensing range would have an upper limit and a lower limit);
generating a device control value according to a value of data of each external factor for each of the division ranges (Graefe, para [0055], actuator data provided to actuators would modify sensor data collected by modifying the range of data collection);
transmitting the device control value to each control target device (Graefe, para [0055], ‘configuration subsystem 306 sending commands or instructions, with the assistance of sensor interface subsystem 310, to adapt or adjust sensing parameters such as a viewing area’, sensor interface subsystem 310 transmits actuator data for sensor reconfiguration);
calculating a score indicating a reward obtained from a control result of each control target device (Graefe, para [0104-1055], terms within fitness function equation attribute a reward for coverage of sensors);
storing each learning data indicated by the device control value and the score as a control result thereof in a learning data DB for each device control factor pattern indicating the device control value corresponding to the division range of each external factor (Graefe, para [0050-51], stores and accesses data for the IoT network, SAS(Sensor Arrangement Service)); Graefe, para [0103], parameters and algorithms implemented for configuration subsystem 306; Graefe, para [0055], range);
acquiring the learning data, which is only changed a specified external factor, after one of external factors is specified, and other external factors excluding the external factor, and the device control factor pattern are fixed, from the learning data DB (Graefe, para [0099], with regards to fig 4, elements 414 and 416, sensor arrange is initialized and implemented, interpreted as “fixed” within a state. When a trigger is detected, such as a change in traffic conditions, sensor arrangement reconfiguration implemented. A change in detected data, such as traffic conditions is interpreted as an external factor);
extracting the score of the learning data (Graefe, para [0099 and 103], with regards to 414, sensor arrangement that is based on an output of a ML model is initialized and the output score is compared to a threshold. The sensor arrangement is based on a ML model fed (interpreted as an initial training pass) with initial detection capabilities);
Graefe does not disclose:
calculating a predetermined impurity of the specified external factor by determining which of divided classes divided into predetermined classes according to a level of the score, calculates the impurity in the same device control factor pattern for each of the external factors;
extracting top N external factors having the calculated large impurity;
extracting P external factors in descending order of sum of the number of appearances from the top N external factors extracted in a predetermined M or more device control factor patterns to be a constitution element of a situation as a factor affecting reward variation;
dividing extracted each value of the P external factors into predetermined Q range widths;
constituting a decision tree that branches in the order of extraction;
defining each of final branch points in the constructed decision tree as a classification that is one of the situations; and
generating a learning model for each of the classifications by performing reinforcement learning so as to satisfy a predetermined reward using the defined learning data for each of the classifications,
wherein generating the learning model for each of the classifications comprises:
collecting learning data by generating the device control value and by updating the learning model for each classification, until the predetermined reward is satisfied.
Jagannath discloses:
calculating a predetermined impurity of the specified external factor by determining which of divided classes divided into predetermined classes according to a level of the score, calculates the impurity in the same device control factor pattern for each of the external factors (Jagannath, page 9, col 2, para 3, calculates Gini impurities for leaf node (eqn 15) and tree (eqn 16));
extracting top N external factors having the calculated large impurity (Jagannath, page 9, col 2, para 3, tree impurity calculation represents a large impurity);
extracting P external factors in descending order of sum of the number of appearances from the top N external factors extracted in a predetermined M or more device control factor patterns to be a constitution element of a situation as a factor affecting reward variation (Jagannath, page 9, col 2, para 3, determining the entropy. High entropy indicates a mix of different classes (high impurity), while low entropy indicates a purer set with classes that are more uniform. Decision trees use entropy to select the best features to split the data by choosing the split that results in the largest decrease in entropy);
dividing extracted each value of the P external factors into predetermined Q range widths (Jagannath, page 9, col 2, para 3, splits minimize measure of impurity);
constituting a decision tree that branches in the order of extraction (Jagannath, page 9, col 2, para 3, decision tree);
defining each of final branch points in the constructed decision tree as a classification that is one of the situations (Jagannath, page 9, col 2, last paragraph, applying a bagging algorithm for classification); and
generating a learning model for each of the classifications by performing reinforcement learning so as to satisfy a predetermined reward using the defined learning data for each of the classifications,
wherein generating the learning model for each of the classifications comprises:
collecting learning data by generating the device control value and by updating the learning model for each classification, until the predetermined reward is satisfied (Jagannath, page 9, col 2, last paragraph, Bagged algorithm is Random Forest, which are bagged decision trees with the following steps:
- Sample m datatsets D 1 , . . . , D m from D with replacement.
- For each D i train a decision tree classifier h i ( · ) to the maximum
depth and when splitting the tree only consider a subset
of features k .
-The ensemble classifier is then the mean output decision).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the generic machine learning model implemented in an IoT system to include a random forest model based on the teachings of Jagannath. The motivation for doing so would have been classification and regression modeling with a light weight solution (Jaggarnath, page 9, col 2, first paragraph).
Claims 5 and 7 are substantially similar to claim 1 and are thus similarly rejected.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Graefe United States Patent Application Publication US 2019/0222652 in view of Jaggannath et al “Machine Learning for Wireless Communications in the Internet of Things: A Comprehensive Survey” Ad Hoc Networks 93 (7/18/19) in further view of Vaughan United States Patent Application Publication US 2019//0019581.
Regarding claim 2, Graefe in view of Jaggannath discloses the device control value generation device according to claim 1. Graefe in view of Jaggannath does not disclose the additional limitations of the present claim.
Vaughan discloses wherein a processor is configured to extract the external factor that is a constitution element of the situation and executes a definition of the classification at predetermined time intervals. (Vaughan, para [0203], monitoring with periodic reassessment for applications that require time between events).
Before the time of the effective fining date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the classification to be at predetermined time intervals. The motivation for doing so would have been to reassess external data that may require monitoring over a period of time (Vaughan, para [0203]).
Claim(s) 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Graefe United States Patent Application Publication US 2019/0222652 in view of Jaggannath et al “Machine Learning for Wireless Communications in the Internet of Things: A Comprehensive Survey” Ad Hoc Networks 93 (7/18/19) in further view of Ebrahami Afrouzi United States Patent Application Publication US 2020/0225673.
Regarding claim 3, Graefe in view of Jaggannath discloses the device control value generation device according to claim 1. Graefe in view of Jaggannath does not disclose the additional limitations of the present claim.
Ebrahimi Afrouzi discloses wherein the processor is configured to determine a location characteristic indicating a factor affecting the unknown or unmeasured reward other than the external factor has changed when a score of learning data in the same classification does not satisfy the predetermined reward continuously for a first predetermined period or longer in an operation stage after the score satisfies the predetermined reward (Ebrahimi Afrouzi, para [0370-371] maximizing reward function. ‘While executing the movement path, in some embodiments, rewards may be assigned by the processor as the robot takes actions to transition between states and uses the net cumulative reward to evaluate a particular movement path comprised of actions and states.’), wherein when the processor determines that the score does not satisfy the predetermined reward continuously for a first predetermined period or longer, the processor is configured to delete learning data before the first predetermined period, and to update the learning model for each classification. (Ebrahimi Afrouzi, para [0370-371] may minimize the total cost function by modifying zones of the environment by, for example, removing, adding, shrinking, expanding, moving and switching the order of coverage of zones).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the classification to include additional information about the reward. The motivation for doing so would have been determine optimal (e.g., locally or globally) division and coverage of the environment (Ebrahimi Afrouzi, para [0371]).
Regarding claim 4, Graefe in view of Jaggannath in further view of Ebrahimi Afrouzi discloses the device control value generation device according to claim 1. Ebrahimi Afrouzi additionally discloses
4. (Currently Amended) The device control value generation device according to claim 3, wherein the processor is configured to issue an alert, when the learning model is updated more than a predetermined number of times in a second predetermined period due to a determination that the location characteristic has changed, and when disturbance fluctuation due to an unknown external factor occurs (Ebrahimi Afrouzi, para [0370-371], iteratively evolve to become more efficient by choosing transitions/paths that result in most favorable outcomes and by avoiding situations that previously resulted in low net reward; Ebrahimi Afrouzi, para [0397], when a path is altered because of a detected object, processor sends alert).
Before the time of the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the classification to include additional information about the reward. The motivation for doing so would have been determine optimal (e.g., locally or globally) division and coverage of the environment (Ebrahimi Afrouzi, para [0371]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HOPE C SHEFFIELD whose telephone number is (303)297-4265. The examiner can normally be reached Monday-Friday, 9:00 am-5:00pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached at (571)270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HOPE C SHEFFIELD/ Primary Examiner, Art Unit 2141