Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-22 are method claims. Therefore, claims 1-22 are directed to either a process, machine, manufacture or composition of matter.
With respect to claim 1:
Step 2A – Prong 1:
…
…
identifying, … in the first set of visual data, a machine placement candidate point associated with identifying the threshold point; (mental process – a person can manually identify in the first set of visual data, a machine placement candidate point associated with identifying the threshold point with the assistance of a pen/paper.)
identifying, based on the machine placement candidate point, a set of baseline confidence values via a baseline uncertainty model; (mental process – a person can manually identify a set of baseline confidence values via a baseline uncertainty model with the assistance of a pen/paper.)
determining, … based on the set of baseline confidence values, a state space, wherein the determining comprises determining differences between successive baseline confidences values in the set of baseline confidence values; (mental process – a person can manually determine a state space, wherein the determining comprises determining differences between successive baseline confidences values in the set of baseline confidence values with the assistance of a pen/paper.)
…
…
(iii) comparing the baseline confidence values with locations associated with the additional threshold points, (mental process – a person can manually compare the baseline confidence values with locations associated with the additional threshold points with the assistance of a pen/paper.)
(iv) identifying an amount of error associated with a window of the additional threshold points, (mental process – a person can manually identify an amount of error associated with a window of the additional threshold points with the assistance of a pen/paper.)
(v) generating reward values based on the identified amount of error, and (mental process – a person can manually generate reward values based on the identified amount of error with the assistance of a pen/paper.)
…
and identifying, … a visual feature in a second set of visual data, the visual feature being associated with the classification task. (mental process – a person can manually identify a visual feature in a second set of visual data, the visual feature being associated with the classification task with the assistance of a pen/paper.)
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
A method comprising: providing, by a processing device, a first set of visual data; (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
receiving, by the processing device, user input associated with identifying a threshold point in the first set of visual data, the threshold point being associated with a classification task; (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
… by the processing device (mere instructions to apply the exception using a generic computer component – processor applies exception) via a machine learning model, … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to identify a candidate point.);
…
… by the processing device, … (mere instructions to apply the exception using a generic computer component – processor applies exception)
training, by the processing device, the machine learning model based on the determined state space, wherein training comprises (i) identifying, via the machine learning model, in one or more subsequent sets of visual data additional threshold points associated with the classification task, (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to identify points associated with a classification task.);
(ii) receiving user feedback indicating an accuracy associated with each of the additional threshold points, (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
…
…
…
(vi) configuring the machine learning model based on the generated reward values; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine based on reward values.);
… by the processing device, … (mere instructions to apply the exception using a generic computer component – processor applies exception) via the trained machine learning model, … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to identify a visual feature.);
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
A method comprising: providing, by a processing device, a first set of visual data; (MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the visual data is merely transmitted/provided). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
receiving, by the processing device, user input associated with identifying a threshold point in the first set of visual data, the threshold point being associated with a classification task; (MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the user input is merely received). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
… by the processing device (mere instructions to apply the exception using a generic computer component – processor applies exception) via a machine learning model, … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to identify a candidate point.);
…
… by the processing device, … (mere instructions to apply the exception using a generic computer component – processor applies exception)
training, by the processing device, the machine learning model based on the determined state space, wherein training comprises (i) identifying, via the machine learning model, in one or more subsequent sets of visual data additional threshold points associated with the classification task, (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to identify points associated with a classification task.);
(ii) receiving user feedback indicating an accuracy associated with each of the additional threshold points, (MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the user input is merely received). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
…
…
…
(vi) configuring the machine learning model based on the generated reward values; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine based on reward values.);
… by the processing device, … (mere instructions to apply the exception using a generic computer component – processor applies exception) via the trained machine learning model, … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the machine learning engine to identify a visual feature.);
With respect to claim 2:
Step 2A – Prong 1:
The method of claim 1, wherein providing the first set of visual data comprises displaying an image. (mental process – a person can recognize that the first set of visual data comprises displaying an image.)
With respect to claim 3:
Step 2A – Prong 1:
The method of claim 1, wherein providing the first set of visual data comprises displaying a graph. (mental process – a person can recognize that providing the first set of visual data comprises displaying a graph.)
With respect to claim 4:
Step 2A – Prong 1:
The method of claim 1, wherein the baseline uncertainty model comprises a naive Bayes model. (mental process – a person can recognize that the baseline uncertainty model comprises a naive Bayes model.)
With respect to claim 5:
Step 2A – Prong 1:
The method of claim 1, wherein the classification task comprises identifying a boundary between a high value and a low value. (mental process – a person can recognize that the classification task comprises identifying a boundary between a high value and a low value.)
With respect to claim 6:
Step 2A – Prong 1:
The method of claim 1, wherein the second set of visual data comprises an image. (mental process – a person can recognize that the second set of visual data comprises an image.)
With respect to claim 7:
Step 2A – Prong 1:
The method of claim 1, wherein the second set of visual data comprises a graph. (mental process – a person can recognize that the second set of visual data comprises a graph.)
With respect to claim 8:
Step 2A – Prong 1:
The method of claim 1, wherein identifying the visual feature in the second set of visual data comprises identifying an edge between two regions in the second set of visual data. (mental process – a person can recognize that identifying the visual feature in the second set of visual data comprises identifying an edge between two regions in the second set of visual data.)
With respect to claim 9:
Step 2A – Prong 1:
The method of claim 1, further comprising determining one or more discretization values of differences between successive baseline confidences values in the set of baseline confidence values. (mental process – a person can recognize that determining one or more discretization values of differences between successive baseline confidences values in the set of baseline confidence values.)
With respect to claim 10:
Step 2A – Prong 1:
The method of claim 1, further comprising performing a water-based operation based on the identified visual feature in the second set of visual data. (mental process – a person can recognize that performing a water-based operation based on the identified visual feature in the second set of visual data.)
With respect to claim 11:
Step 2A – Prong 1:
The method of claim 1, wherein the visual feature is associated with an edge between two regions in the visual data. (mental process – a person can recognize that the visual feature is associated with an edge between two regions in the visual data.)
With respect to claim 12:
Step 2A – Prong 1:
The method of claim 1, wherein the classification task is a temporal-based task. (mental process – a person can recognize that the classification task is a temporal-based task.)
With respect to claim 13:
Step 2A – Prong 1:
The method of claim 1, further comprising determining one or more tolerance values indicating a maximum distance a machine placement can be from a user placement to be deemed correct. (mental process – a person can recognize that determining one or more tolerance values indicating a maximum distance a machine placement can be from a user placement to be deemed correct.)
With respect to claim 14:
Step 2A – Prong 1:
The method of claim 13, wherein determining the one or more tolerance values comprises calculating a mean absolute error. (mental process – a person can recognize that determining the one or more tolerance values comprises calculating a mean absolute error.)
With respect to claim 15:
Step 2A – Prong 1:
The method of claim 13, wherein the one or more tolerance values range from 0.02 to 0.20. (mental process – a person can recognize that one or more tolerance values range from 0.02 to 0.20.)
With respect to claim 16:
Step 2A – Prong 1:
The method of claim 1, wherein determining the state space comprises using a Markov Decision Process framework. (mental process – a person can recognize that determining the state space comprises using a Markov Decision Process framework.)
With respect to claim 17:
Step 2A – Prong 1:
The method of claim 16, wherein the Markov Decision Process framework uses Q-Learning by estimating a reward value at a state action pair to find an optimal policy associated with the state space. (mental process – a person can recognize that Markov Decision Process framework uses Q-Learning by estimating a reward value at a state action pair to find an optimal policy associated with the state space.)
With respect to claim 18:
Step 2A – Prong 1:
The method of claim 1, wherein generating reward values comprises using a reward function that produces high reward values responsive to producing confidence values that are on average closer to an accuracy of a classifier within the window. (mental process – a person can recognize that generating reward values comprises using a reward function that produces high reward values responsive to producing confidence values that are on average closer to an accuracy of a classifier within the window.)
With respect to claim 19:
Step 2A – Prong 1:
The method of claim 18, wherein the accuracy of the classifier is based on precision feedback from a user. (mental process – a person can recognize that accuracy of the classifier is based on precision feedback from a user.)
With respect to claim 20:
Step 2A – Prong 1:
The method of claim 1, further comprising improving the baseline confidence values by aligning a probability of accurate classification based on feedback from a user. (mental process – a person can recognize that baseline confidence values by aligning a probability of accurate classification based on feedback from a user.)
With respect to claim 21:
Step 2A – Prong 1:
The method of claim 1, wherein the first set of visual data is streaming data. (mental process – a person can recognize that the first set of visual data is streaming data.)
With respect to claim 22:
Step 2A – Prong 1:
The method of claim 1, wherein the second set of visual data is streaming data. (mental process – a person can recognize that the second set of visual data is streaming data.)
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-22 are rejected under 35 U.S.C. 103 as being unpatentable over Michael et al. (“On Interactive Machine Learning and the Potential of Cognitive Feedback”) hereinafter known as Michael in view of Taylor et al. (US20190236458A1) hereinafter known as Taylor.
Regarding independent claim 1, Michael teaches:
A method comprising: providing, by a processing device, a first set of visual data; (Michael [Page 3, Col. 2, last paragraph]: “bodies of water and other land cover are digitized from remotely sensed images” Michael teaches that remotely sensed image data are processed by digitization.)
receiving, by the processing device, user input associated with identifying a threshold point in the first set of visual data, the threshold point being associated with a classification task; (Michael [Page 4, Col. 1, Paragraph 1]: “For each vertex presented, the analyst may either correct its placement by clicking and dragging it to an appropriate location” Michael teaches that the user/analyst may input and adjust a threshold point during the task of assigning a point on a contour.)
identifying, by the processing device via a machine learning model, in the first set of visual data, a machine placement candidate point associated with identifying the threshold point; (Michael [Page 4, Col. 1, Paragraph 1]: “In each iteration, the machine guesses the placement of a certain number of vertices of the contour and presents them to the analyst for verification” Michael teaches that the machine first places vertices of the contour as a candidate for the analyst to verify.)
identifying, based on the machine placement candidate point, a set of baseline confidence values via a baseline uncertainty model; (Michael [Page 4, Col. 1, Paragraph 1]: “an uncertainty model is used to estimate the probability of incorrect vertex placement and limit each iteration to around 2 incorrectly placed vertices.” Michael teaches baseline confidence values, as the probability of incorrect vertex placement is estimated.)
…
training, by the processing device, the machine learning model based on the determined state space, wherein training comprises (i) identifying, via the machine learning model, in one or more subsequent sets of visual data additional threshold points associated with the classification task, (Michael [Page 4, Col. 1, Paragraph 1]: “In each iteration, the machine guesses the placement of a certain number of vertices of the contour and presents them to the analyst for verification” Michael teaches that the machine first places vertices of the contour as a candidate for the analyst to verify.)
(ii) receiving user feedback indicating an accuracy associated with each of the additional threshold points, (Michael [Page 4, Col. 1, Paragraph 1]: “For each vertex presented, the analyst may either correct its placement by clicking and dragging it to an appropriate location” Michael teaches that the user/analyst may input and adjust a threshold point during the task of assigning a point on a contour.)
(iii) comparing the baseline confidence values with locations associated with the additional threshold points, (Michael [Page 4, Col. 1, Paragraph 1]: “In order to prevent inducing too high of a cognitive load on the analyst, an uncertainty model is used to estimate the probability of incorrect vertex placement and limit each iteration to around 2 incorrectly placed vertices.” Michael teaches that the confidence of the vertex placement is calculated. Michael uses the probability of incorrect vertex placement to ensure that no more than 2 vertices are incorrect most of the time.)
…
…
…
and identifying, by the processing device, via the trained machine learning model, a visual feature in a second set of visual data, the visual feature being associated with the classification task. (Michael [Page 6, Col. 2, Paragraph 2]: “Though the shoreline may be spotted by an analyst clearly in the fourth image, the classifier overfit to spatial features and thus incorrectly identified the shoreline” Michael teaches that the shoreline, a visual feature, was classified in a test run.)
Michael does not explicitly teach:
determining, by the processing device, based on the set of baseline confidence values, a state space, wherein the determining comprises determining differences between successive baseline confidences values in the set of baseline confidence values;
…
…
…
(iv) identifying an amount of error associated with a window of the additional threshold points,
(v) generating reward values based on the identified amount of error, and
(vi) configuring the machine learning model based on the generated reward values;
However, Taylor teaches:
determining, by the processing device, based on the set of baseline confidence values, a state space, wherein the determining comprises determining differences between successive baseline confidences values in the set of baseline confidence values; (Taylor [0026]: “the confidence data values are generated using a dynamic temporal difference confidence measurement based on the relation … α is an update parameter” Taylor teaches a dynamic temporal difference confidence measurement, which when the states are combined are a set of states. Each dynamic temporal difference confidence measurement is based on the current and successive states.)
…
…
…
(iv) identifying an amount of error associated with a window of the additional threshold points, (Taylor [0195]: “Use a sliding window of 10×10 to scan neighbourhood positions and calculate the average “CP(s)” [confidence of prior knowledge] within that sliding window.” Taylor teaches an amount of error/confidence associated with a window of neighborhood positions/points.)
(v) generating reward values based on the identified amount of error, and (Taylor [0137]: “reward function is leveraging the confidence” Taylor teaches that the reward is leveraging the confidence, which is based on the amount of error.)
(vi) configuring the machine learning model based on the generated reward values; (Taylor [0024]: “The internal policy function maintained by the machine learning model is updated based at least on the observed reward outcome” Taylor teaches that the policy function of the model is updated based on the observed rewards.)
Michael and Taylor are in the same field of endeavor as the present invention, as the references are directed to machine learning systems that use human feedback and confidence/uncertainty measurements through reinforcement learning. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine iteratively adjusting machine placed vertices with an agent as taught in Michael with using a window of error and basing reward values on this error as taught in Taylor. Taylor provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Michael to include teachings of Taylor because the combination would allow for human input to make sure that the vertices are correctly classifying the input data. This has the potential benefit of more accurately training the machine learning model, as the model is given a few points of ground truth in each iteration.
Regarding dependent claim 2, Michael and Taylor teach:
The method of claim 1, wherein providing the first set of visual data comprises displaying an image. (Michael [Page 3, Col. 2, last paragraph]: “bodies of water and other land cover are digitized from remotely sensed images” Michael teaches that remotely sensed image data are processed by digitization.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 3, Michael and Taylor teach:
The method of claim 1, wherein providing the first set of visual data comprises displaying a graph. (Taylor [Figure 2]: Taylor displays a graph comparison of learning curves of DRoP, CHAT and baseline RL in Cartpole.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 4, Michael and Taylor teach:
The method of claim 1, wherein the baseline uncertainty model comprises a naive Bayes model. (Taylor [0069]: “Three confidence models are provided: a Decision Tree, a Gaussian Cluster, and a Neural Network” Taylor teaches that the uncertainty/confidence model includes a Gaussian cluster.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 5, Michael and Taylor teach:
The method of claim 1, wherein the classification task comprises identifying a boundary between a high value and a low value. (Michael [Page 6, Col. 2, Paragraph 2]: “Though the shoreline may be spotted by an analyst clearly in the fourth image, the classifier overfit to spatial features and thus incorrectly identified the shoreline” Michael teaches that the shoreline, a visual feature, was classified in a test run. The shoreline has water and land, which should have polarizing values.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 6, Michael and Taylor teach:
The method of claim 1, wherein the second set of visual data comprises an image. (Michael [Page 3, Col. 2, last paragraph]: “bodies of water and other land cover are digitized from remotely sensed images” Michael teaches that remotely sensed image data are processed by digitization.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 7, Michael and Taylor teach:
The method of claim 1, wherein the second set of visual data comprises a graph. (Taylor [Figure 2]: Taylor displays a graph comparison of learning curves of DRoP, CHAT and baseline RL in Cartpole.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 8, Michael and Taylor teach:
The method of claim 1, wherein identifying the visual feature in the second set of visual data comprises identifying an edge between two regions in the second set of visual data. (Michael [Page 3, Col. 2, last paragraph]: “bodies of water and other land cover are digitized from remotely sensed images” Michael teaches that bodies of water are digitized and analyzed. Bodies of water have a contour/edge that separates two or more land/water regions.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 9, Michael and Taylor teach:
The method of claim 1, further comprising determining one or more discretization values of differences between successive baseline confidences values in the set of baseline confidence values. (Taylor [0026]: “the confidence data values are generated using a dynamic temporal difference confidence measurement based on the relation … α is an update parameter” Taylor teaches a dynamic temporal difference confidence measurement, which when the states are combined are a set of states. Each dynamic temporal difference confidence measurement is based on the current and successive states.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 10, Michael and Taylor teach:
The method of claim 1, further comprising performing a water-based operation based on the identified visual feature in the second set of visual data. (Michael [Page 3, Col. 2, last paragraph]: “bodies of water and other land cover are digitized from remotely sensed images” Michael teaches that bodies of water are digitized, which is a water-based operation.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 11, Michael and Taylor teach:
The method of claim 1, wherein the visual feature is associated with an edge between two regions in the visual data. (Michael [Page 3, Col. 2, last paragraph]: “bodies of water and other land cover are digitized from remotely sensed images” Michael teaches that bodies of water are digitized and analyzed. Bodies of water have a contour/edge that separates two or more land/water regions.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 12, Michael and Taylor teach:
The method of claim 1, wherein the classification task is a temporal-based task. (Michael [Page 5, Col. 1, Paragraph 1]: “allows for a user to annotate a video frame with an arbitrary label, for instance jump. Then, using trajectory information extracted from the video, the machine trains on the given label and presents classification results both at the level of the current video and a database of numerous animal videos” Michael teaches that a classification is made on the level of the current video, which is temporal as each frame is based on a time.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 13, Michael and Taylor teach:
The method of claim 1, further comprising determining one or more tolerance values indicating a maximum distance a machine placement can be from a user placement to be deemed correct. (Michael [Page 4, Col. 1, Paragraph 1]: “In order to prevent inducing too high of a cognitive load on the analyst, an uncertainty model is used to estimate the probability of incorrect vertex placement and limit each iteration to around 2 incorrectly placed vertices.” Michael teaches that the confidence of the vertex placement is calculated. Michael uses the probability of incorrect vertex placement to ensure that no more than 2 vertices are incorrect most of the time. This indicates a maximum distance that may be variable per iteration, as the standard for which probability is unacceptable after the second least accurate vertex may depend on the iteration.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 14, Michael and Taylor teach:
The method of claim 13, wherein determining the one or more tolerance values comprises calculating a mean absolute error. (Michael [Page 8, Col. 2, Paragraph 1]: “performing best-fit optimization to prior iterations” Michael teaches using best-fit optimization to prior iterations to calculate the uncertainty model. This calculates the mean absolute error mathematically as it is a line of best fit of previous data.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 15, Michael and Taylor teach:
The method of claim 13, wherein the one or more tolerance values range from 0.02 to 0.20. (Taylor [0193, Table 2]: Taylor teaches that the converged reuse frequency has a plus/minus of values ranging from 0.02 to 0.20, with 0.04 as an example.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 16, Michael and Taylor teach:
The method of claim 1, wherein determining the state space comprises using a Markov Decision Process framework. (Taylor [0059]: “A Markov decision process (MDP) is common formulation of the RL problem” Taylor teaches that the formulation of the RL problem is commonly represented as a Markov decision process.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 17, Michael and Taylor teach:
The method of claim 16, wherein the Markov Decision Process framework uses Q-Learning by estimating a reward value at a state action pair to find an optimal policy associated with the state space. (Taylor [0061]: “Q-learning: Q(s, a)←Q(s, a)+α[r+γ max Q(s′, a′)−Q(s, a)] Given a library L={II1, . . . , IIn} of n past optimal polices” Taylor teaches using Q learning in reinforcement learning to find an optimal policy.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 18, Michael and Taylor teach:
The method of claim 1, wherein generating reward values comprises using a reward function that produces high reward values responsive to producing confidence values that are on average closer to an accuracy of a classifier within the window. (Taylor [0137]: “reward function is leveraging the confidence” Taylor teaches that the reward is leveraging the confidence, which is based on the amount of error.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 19, Michael and Taylor teach:
The method of claim 18, wherein the accuracy of the classifier is based on precision feedback from a user. (Michael [Page 4, Col. 1, Paragraph 1]: “For each vertex presented, the analyst may either correct its placement by clicking and dragging it to an appropriate location” Michael teaches that the user/analyst may input and adjust a threshold point during the task of assigning a point on a contour.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 20, Michael and Taylor teach:
The method of claim 1, further comprising improving the baseline confidence values by aligning a probability of accurate classification based on feedback from a user. (Michael [Page 8, Col. 1, Paragraph 2]: “the analyst will inform the machine that the workload is too much to handle, and the machine may modify its uncertainty model accordingly (e.g. by adjusting weighting or performing best-fit optimization to prior iterations)” Michael teaches that the feedback from the analyst results in the uncertainty model adjusting by using optimization from prior iterations.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 21, Michael and Taylor teach:
The method of claim 1, wherein the first set of visual data is streaming data. (Michael [Page 2, Col. 2, Paragraph 2]: “The data on which the analyst must perform a task may either be completely available in a database or sequentially available as a stream.” Michael teaches that the data that the analyst performs the task on may be sequentially available as a stream.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 22, Michael and Taylor teach:
The method of claim 1, wherein the second set of visual data is streaming data. (Michael [Page 2, Col. 2, Paragraph 2]: “The data on which the analyst must perform a task may either be completely available in a database or sequentially available as a stream.” Michael teaches that the data that the analyst performs the task on may be sequentially available as a stream.)
The reasons to combine are substantially similar to those of claim 1.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYU HYUNG HAN whose telephone number is (703) 756-5529. The examiner can normally be reached on MF 9-5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kyu Hyung Han/
Examiner
Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights