DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This communication is in response to the Application Filed on 12/20/2024.
Claims 1–21 are pending in this application.
Drawings
The drawings are objected to because the unlabeled rectangular box(es) shown in the drawings should be provided with descriptive text labels. FIG. 1 has the following elements 20, 30, 40, 50, 60 and 70. According to the specification the elements represent ¶ [0025], “the assessment dataset module 20”, ¶ [0025], “The baseline dataset module 30”, ¶ [0025], “the database module 40”, ¶ [0026], “the comparison module 50”, ¶ [0026], “the output module 60”, and ¶ [0027], “the routing module 70”, respectively. FIG. 2 has the following elements 110, 120A, 120B, 120C, 120D and 130. According to the specification the elements represent ¶ [0040], “From Fig. 2, it can be seen that the dataset module 100 includes an input submodule 110, multiple pre-processing submodules 120A, 120B, 120C, 120D, and a measuring submodule 130”. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 1, 18 and 19 are objected to because of the following informalities:
Claim 18 lines 9-14 recite the limitations “e) comparing metrics obtained in step c) with metrics for elements of said at least one baseline dataset obtained in step d); e) determining which elements of said assessment dataset conform to at least one predetermined condition, said at least one predetermined condition being based on results of step e); f) autonomously executing at least one predetermined action for said elements of said first dataset that conform to said at least one predetermined condition”. There seems to be two steps that are enumerated to label “e)”. The second occurrence at line 11 should be “f)” and the step “f)” at line 14 should be “g)”. So, all in all, perhaps these limitations ought to be “e) comparing metrics obtained in step c) with metrics for elements of said at least one baseline dataset obtained in step d); f) determining which elements of said assessment dataset conform to at least one predetermined condition, said at least one predetermined condition being based on results of step e); g) autonomously executing at least one predetermined action for said elements of said first dataset that conform to said at least one predetermined condition”.
Claims 1, 18 and 19 are missing “and” phrase to complete the list of steps. Specifically for claim 1, Pg. 1, ln. 9, step “d)” and Pg. 1, ln. 12, after the first wherein clause, Claim 18, Pg. 4, ln. 13, step “e)” and Pg. 4, ln. 16, after the first wherein, and Claim 19, Pg. 5, ln. 7, step “d)” the examiner suggests ending limitations with “and” to make it complete.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process (concept performed in a human mind, including as observation, evaluation, judgment, opinion, organizing human activity and mathematical concepts and calculations). The independent claims 1, 18 and 19 recite a method. This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved .The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally and no additional features in the claims would preclude them from being performed as such except for the generic computer elements at high level of generality (i.e., processor, memory).
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that the independent claims 1, 18 and 19 are directed to an abstract idea as shown below:
STEP 1: Do the claims fall within one of the statutory categories? YES. Independent claims 1, 18 and 19 are directed to a method.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? YES, the claims are directed toward a mental process (i.e. abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
Independent claims 1, 18 and 19 comprise a mental process that can be practicably performed in the human mind (or generic computers or components configured to perform the method) and, therefore, an abstract idea.
Regarding independent claim 1: the limitations recite:
a) receiving a first dataset for assessment, said first dataset having multiple elements (data gathering, insignificant pre-solution);
b) determining at least one metric for at least one of said multiple elements of said first dataset (mental process including observation and evaluation);
c) comparing metrics determined in step b) with metrics for elements of at least one second dataset (mental process including observation and evaluation);
d) determining which elements of said first dataset conform to at least one predetermined condition, said at least one predetermined condition being based on results of step c) (mental process including observation and evaluation);
e) autonomously executing at least one predetermined action for said elements of said first dataset that conform to said at least one predetermined condition (mental process including observation and evaluation);
wherein said first dataset and said at least one second dataset comprise visual data (data gathering, insignificant pre-solution);
wherein said first dataset comprises visual data for annotation to result in annotated data (mental process including observation and evaluation), said annotated data being for use in machine learning applications (insignificant post-solution).
Regarding independent claim 18: the limitations recite:
a) receiving said assessment dataset for assessment, said first dataset having multiple elements (data gathering, insignificant pre-solution);
b) receiving at least one baseline dataset (data gathering, insignificant pre-solution);
c) determining at least one metric for at least one of said multiple elements of said assessment dataset (mental process including observation and evaluation);
d) determining said at least one metric for elements of said at least one baseline dataset (mental process including observation and evaluation);
e) comparing metrics obtained in step c) with metrics for elements of said at least one baseline dataset obtained in step d) (mental process including observation and evaluation);
e) determining which elements of said assessment dataset conform to at least one predetermined condition, said at least one predetermined condition being based on results of step e) (mental process including observation and evaluation);
f) autonomously executing at least one predetermined action for said elements of said first dataset that conform to said at least one predetermined condition (mental process including observation and evaluation);
wherein said assessment dataset and said at least one baseline dataset comprise visual data; wherein said assessment dataset comprises visual data for annotation to result in annotated data (mental process including observation and evaluation), said annotated data being for use in machine learning applications (insignificant post-solution).
Regarding independent claim 19: the limitations recite:
a) receiving said dataset, said dataset having multiple elements (data gathering, insignificant pre-solution);
b) determining at least one metric for at least one of said multiple elements of said dataset (data gathering, insignificant pre-solution);
c) determining at least one predetermined limit for said at least one metric (data gathering, insignificant pre-solution);
d) determining which elements of said dataset conform to at least one predetermined condition, said at least one predetermined condition being based said at least one predetermined limit (data gathering, insignificant pre-solution);
e) autonomously executing at least one predetermined action for said elements of said dataset that conform to said at least one predetermined condition (data gathering, insignificant pre-solution);
wherein said dataset comprises visual data for annotation to result in annotated data (data gathering, insignificant pre-solution), said annotated data being for use in machine learning applications (insignificant post-solution).
These limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
As such, a person could mentally observe two different piles of images with one pile representing baseline images and compare the brightness of the image, the color vibrance, or sharpness of the images from one pile to the images of the baseline pile and do an action such as write a label or text on the image based on how it’s acceptable or unacceptable.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Independent claims 1, 18 and 19 do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
These limitations are recited at a high level of generality (i.e. as a general action or change being taken based on the results of the acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity. Further, the claims are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Independent claims 1, 18 and 19 do not recite any additional elements that are not well-understood, routine or conventional. The use of a generic computer elements are routine, well-understood and conventional process that is performed by computers.
Thus, since independent claims 1, 18 and 19 are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that independent claims 1, 18 and 19 are not eligible subject matter under 35 U.S.C 101.
Regarding claims 2, 3, 5–7, 11–17, 20 and 21: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitations merely recite data gathering and insignificant pre-solution.
Regarding claims 4 and 8–10: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitations merely recite mental process than be done in the human mind and observation.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1, 2, 5, 8–15 and 17–19 are rejected under 35 U.S.C. 103 as being unpatentable over Kushnir et al. (US 20210142168 A1, hereafter, "Kushnir") in view of Hazard et al. (US 20210326652 A1, hereafter, "Hazard").
Regarding claim 1, Kushnir discloses a method for assessing a first dataset relative to at least one second dataset (See Kushnir, [Abstract], for each unlabeled input, a first distance of the unlabeled input to the labeled inputs of the training data set and a second distance of the unlabeled input to other unlabeled inputs of the pool data set. Note: Examiner is interpreting the first dataset as the unlabeled inputs from the pool set and the second dataset as the labeled input of the training dataset), the method comprising:
a) receiving a first dataset for assessment, said first dataset having multiple elements (See Kushnir, ¶ [0055], The apparatus 102 stores a pool data set 110 of unlabeled inputs that are not yet associated with a label. Note: the unlabeled inputs of the pool dataset are being interpreted as the first dataset);
[b) determining at least one metric for at least one of said multiple elements of said first dataset;
c) comparing metrics determined in step b)] with [metrics] for elements of at least one second dataset (See Kushnir, ¶ [0070], The processing circuitry 116 may be configured to select the position of each labeled input 212 and unlabeled input 214 according to the features 304-1, 304-2 of each input from a layer of a neural network 106, and/or to evaluate the unlabeled inputs 218 within the feature space 302. The processing circuitry 116 may be configured to determine a first distance 502, for example, as a distance between the unlabeled input 218 of the pool data set 110 and one or more labeled inputs 212 of the training data set 108. Note: Examiner is interpreting the second dataset as the labeled inputs 212. Examiner interprets the distance as comparing the datasets);
d) determining which elements of said first dataset conform to at least one predetermined condition, said at least one predetermined condition being based on results of step c) (See Kushnir, ¶ [0072], As a first such example, the processing circuitry 116 may be configured to identify the refinement subset 602 as the unlabeled inputs 218 that are distant within the feature space 302 from the labeled inputs 212 (for example, unlabeled inputs 218 having a first distance 502 that is above a distance threshold). The processing circuitry 116 may be configured to include such unlabeled inputs 218 in the refinement subset 602, for example, due to substantial dissimilarity between the unlabeled input 218 and the labeled inputs 212. Note: Examiner interprets the dissimilarity based on the distance as the predetermined condition);
e) autonomously executing at least one predetermined action for said elements of said first dataset that conform to said at least one predetermined condition (See Kushnir, ¶ [0074], In some example embodiments, the processing circuitry 116 may be configured to submit the refinement subset 602 to a labeling process 220 and to receive, in return, a labeled subset 604. See also [FIG. 2], 220 LABELING PROCESS. Note: Examiner is interpreting the submission of the refinement subset to the labeling process as autonomously executing a predetermined action);
wherein said first dataset and said at least one second dataset comprise visual data (See Kushnir, ¶ [0110], For example, the training data set 108 may include labeled inputs 212 including video sequences with labels 216 that indicate the events illustrated within the video sequence, …, The pool data set 110 may include unlabeled inputs 218 including video sequences without labels 216. Note: Examiner interprets the unlabeled input as the first dataset and the labeled input as the second dataset);
wherein said first dataset comprises visual data for annotation to result in annotated data, said annotated data being for use in machine learning applications (See Kushnir, ¶ [0103], In some example embodiments, processing circuitry 116 may be configured to include the refinement subset 602 in further training 608 of a partially trained neural network 606. In some other example embodiments, processing circuitry 116 may be configured to use the labeled subset 604 to retrain 406 a neural network 106. Note: Examiner interprets annotated data as the refinement subset 602. Examiner interprets processing the labeled refinement subset further to train a machine learning model as using the annotated data in a machine learning application).
However, Kushnir fail to teach b) determining at least one metric for at least one of said multiple elements of said first dataset; c) comparing metrics determined in step b) with metrics.
Hazard, working in the same field of endeavor, teaches: b) determining at least one metric for at least one of said multiple elements of said first dataset (See Hazard, ¶ [0191], Entropy of a dataset may be a measure of how compressible or predictable a given feature is, which is generally correlated with the skewedness of the class distribution. ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset. Note: Examiner interprets entropy as the metric for the first dataset. Examiner interprets the first dataset as the synthetic dataset);
c) comparing metrics determined in step b) with metrics (See Hazard, ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset. Note: Examiner interprets entropy as the metric for the first dataset. Examiner interprets the first dataset as the synthetic dataset and comparing the entropy and comparing the metrics).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference to have b) determining at least one metric for at least one of said multiple elements of said first dataset; comparing metrics determined in step b) with metrics based on the method of Hazard’s reference to compare two metric for two datasets. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 1.
Regarding claim 2, Kushnir in view of Hazard teaches the method according to claim 1, wherein said at least one second dataset comprises visual data that has been annotated (See Kushnir, ¶ [0110], For example, the training data set 108 may include labeled inputs 212 including video sequences with labels 216 that indicate the events illustrated within the video sequence).
Regarding claim 5, Kushnir in view of Hazard teaches the method according to claim 1, wherein said visual data is at least one of: images and video (See Kushnir, ¶ [0109], As one such example, the processing circuitry 116 may store or access a training data set as a video sequence of video frames that depict events that are identified by the labeled inputs 212. Note: the dataset can be based on video frames which are visual).
Regarding claim 8, Kushnir in view of Hazard teaches the method according to claim 1, [wherein said predetermined condition is having a metric that is at least equal to or exceeding a predetermined threshold based on metrics for multiple elements for said at least one second dataset].
However, Kushnir fails to teach wherein said predetermined condition is having a metric that is at least equal to or exceeding a predetermined threshold based on metrics for multiple elements for said at least one second dataset.
Hazard, working in the same field of endeavor, teaches: wherein said predetermined condition is having a metric that is at least equal to or exceeding a predetermined threshold based on metrics for multiple elements for said at least one second dataset (See Hazard, ¶ [0224], In some embodiments, the dataset quality metric is compared 640 to one or more thresholds. Comparing 640 the determined 630 dataset quality metric may entail comparing the dataset quality metric to one or more thresholds. For example, a lower threshold and/or a higher threshold may be used for comparison 640 the dataset quality metric).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference wherein said predetermined condition is having a metric that is at least equal to or exceeding a predetermined threshold based on metrics for multiple elements for said at least one second dataset based on the method of Hazard’s reference to compare the metrics of multiple elements to a predetermined threshold to check if it exceeds the threshold. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 8.
Regarding claim 9, Kushnir in view of Hazard teaches the method according to claim 1, [wherein said predetermined condition is having a metric that is at most equal to or below a predetermined threshold based on metrics for multiple elements for said at least one second dataset].
However, Kushnir fails to teach wherein said predetermined condition is having a metric that is at most equal to or below a predetermined threshold based on metrics for multiple elements for said at least one second dataset.
Hazard, working in the same field of endeavor, teaches: wherein said predetermined condition is having a metric that is at most equal to or below a predetermined threshold based on metrics for multiple elements for said at least one second dataset (See Hazard, ¶ [0224], In some embodiments, the dataset quality metric is compared 640 to one or more thresholds. Comparing 640 the determined 630 dataset quality metric may entail comparing the dataset quality metric to one or more thresholds. For example, a lower threshold and/or a higher threshold may be used for comparison 640 the dataset quality metric).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference wherein said predetermined condition is having a metric that is at most equal to or below a predetermined threshold based on metrics for multiple elements for said at least one second dataset based on the method of Hazard’s reference to compare the metrics of multiple elements to a predetermined threshold to check if it is below the threshold. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 9.
Regarding claim 10, Kushnir in view of Hazard teaches the method according to claim 1, wherein said at least one predetermined action comprises routing said elements of said first dataset that conform to said at least one predetermined condition to a specific annotator (See Kushnir, ¶ [0074], In some example embodiments, the processing circuitry 116 may be configured to submit the refinement subset 602 to a labeling process 220 and to receive, in return, a labeled subset 604. ¶ [0072], The processing circuitry 116 may be configured to include such unlabeled inputs 218 in the refinement subset 602, for example, due to substantial dissimilarity between the unlabeled input 218 and the labeled inputs 212. See also [FIG. 2], 220 LABELING PROCESS. Note: Examiner is interpreting the submission of the refinement subset to the labeling process as routing to a specific annotator. Examiner interprets the unlabeled inputs as the first dataset).
Regarding claim 11, Kushnir in view of Hazard teaches the method according to claim 1, wherein said first dataset and said second dataset include at least one of: point clouds, video (See Kushnir, ¶ [0110], For example, the training data set 108 may include labeled inputs 212 including video sequences with labels 216 that indicate the events illustrated within the video sequence, …, The pool data set 110 may include unlabeled inputs 218 including video sequences without labels 216. Note: Examiner interprets the unlabeled input as the first dataset and the labeled input as the second dataset), images, depth maps, radar data, 3D mesh data, 3D voxel data.
Regarding claim 12, Kushnir in view of Hazard teaches the method according to claim 1, [wherein said metrics include metadata associated with said datasets].
However, Kushnir fails to teach wherein said metrics include metadata associated with said datasets.
Hazard, working in the same field of endeavor, teaches: wherein said metrics include metadata associated with said datasets (See Hazard, ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset (e.g., as measured by an entropy comparison metric that is a ratio of the two). Note: Examiner interprets the entropy measurement as metadata).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference wherein said metrics include metadata associated with said datasets based on the method of Hazard’s reference to represent the metric as metadata. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 12.
Regarding claim 13, Kushnir in view of Hazard teaches the method according to claim 1, [wherein said metrics include data associated with said first or second datasets].
However, Kushnir fails to teach wherein said metrics include data associated with said first or second datasets.
Hazard, working in the same field of endeavor, teaches: wherein said metrics include data associated with said first or second datasets (See Hazard, ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset (e.g., as measured by an entropy comparison metric that is a ratio of the two)).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference wherein said metrics include data associated with said first or second datasets based on the method of Hazard’s reference to have the metric associated with both datasets. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 13.
Regarding claim 14, Kushnir in view of Hazard teaches the method according to claim 12, wherein said metadata includes at least one of (Note: Superguide and Plain Meaning of "at least of one of x, y, and z" means including all of the elements. Based on the applicants Specification ¶ [0038], “As an example, if a baseline dataset consists of bright, sunlit daytime images of a beach taken at high tide in Huntington Beach in California (with the location and time of the capture of the images being denoted in metadata) and an assessment dataset is that of bright, sunlit daytime images of a boardwalk taken at noon in Redondo Beach in California (again, with the location and time of capture of the images being denoted in metadata)”. The metadata does not need to include all of the limitations. So for the purpose of examination the Examiner interprets the metadata as needing at least one of the limitations and not all of them):
- details regarding a place of image/dataset capture;
- details regarding a time of image/dataset capture;
- details regarding content of image/dataset;
- details regarding said image/dataset that has been autonomously generated;
- text associated with at least one dataset (See Kushnir, ¶ [0109], As one such example, the processing circuitry 116 may store or access a training data set as a video sequence of video frames that depict events that are identified by the labeled inputs 212. Note: Examiner interprets the labels as text associated with the dataset);
- image descriptor model generated data relating to content in a dataset;
- data resulting from a detection or classification of content in a dataset; and
- model embeddings.
Regarding claim 15, Kushnir in view of Hazard teaches the method according to claim 13, wherein said metrics include at least one of (Note: Superguide and Plain Meaning of "at least of one of x, y, and z" means including all of the elements. Based on the applicants Specification ¶ [0038], “Regarding the set of criteria to be used to determine if one or more datasets or one or more elements of datasets are outliers, this set of criteria can be a set of criteria with a single element (e.g. brightness of an image) or it can be a set of criteria with multiple elements (e.g. contrast of an image, number of pixels in the image, image depth of the image, text associated with the image, and content of the image)”. The metrics do not need to include all of the limitations. So for the purpose of examination the Examiner interprets the metrics as needing at least one of the limitations and not all of them):
- [data that results from a preprocessing of at least one dataset];
- data that results from an implementation of a machine learning algorithm on at least one dataset;
- distribution of classes that are present in one or more datasets; and
- distribution of object sizes that are present within one or more datasets.
However, Kushnir fails to teach data that results from a preprocessing of at least one dataset.
Hazard, working in the same field of endeavor, teaches: data that results from a preprocessing of at least one dataset (See Hazard, ¶ [0224], In some embodiments, comparing the dataset quality metric to one or more thresholds may include scaling the metric, normalizing the metric (and/or the one or more thresholds), and/or other numerical calculations before comparing 640 the dataset quality metric to a threshold. Note: Examiner is interpreting normalizing the metric before comparison as data that results from a preprocessing of at least one dataset).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference based on Hazard’s reference to have the data that results from a preprocessing of at least one dataset. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 15.
Regarding claim 17, Kushnir in view of Hazard teaches the method according to claim 15, [wherein distribution data relating to one or more datasets is automatically determined in a pre-processing step applied to one or more datasets].
However, Kushnir fails to teach wherein distribution data relating to one or more datasets is automatically determined in a pre-processing step applied to one or more datasets.
Hazard, working in the same field of endeavor, teaches: wherein distribution data relating to one or more datasets is automatically determined in a pre-processing step applied to one or more datasets (See Hazard, ¶ [0100], Once a feature is selected 130, then a value for the feature may be determined 150. Determining 150 the value for the feature may be accomplished based on the determined 140 distribution for the feature in the training data (as described elsewhere herein) as a whole (sometimes referred to as the global model). In some embodiments, the value for the features may be determined 150 by determining 140 the distribution for the feature for the local model (e.g., the k nearest neighbors)).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference wherein distribution data relating to one or more datasets is automatically determined in a pre-processing step applied to one or more datasets based on the method of Hazard’s reference to determine distribution data of the dataset. The suggestion/motivation would have been to accurately compare and determine the quality of the dataset to produce more accurate datasets for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 17.
Regarding claim 18, Kushnir discloses a method for assessing an assessment dataset relative to at least one baseline dataset (See Kushnir, [Abstract], for each unlabeled input, a first distance of the unlabeled input to the labeled inputs of the training data set and a second distance of the unlabeled input to other unlabeled inputs of the pool data set. Note: Examiner is interpreting the assessment dataset as the unlabeled inputs from the pool set and the baseline dataset as the labeled input of the training dataset), the method comprising:
a) receiving said assessment dataset for assessment, said first dataset having multiple elements (See Kushnir, ¶ [0055], The apparatus 102 stores a pool data set 110 of unlabeled inputs that are not yet associated with a label. Note: the unlabeled inputs of the pool dataset are being interpreted as the assessment dataset);
b) receiving at least one baseline dataset (See Kushnir, ¶ [0055], The memory 104 of the apparatus 102 stores a training data set 108 including a set of labeled inputs that may be provided to train the neural network 106. Note: Examiner interprets the baseline dataset as the labeled inputs);
[c) determining at least one metric for at least one of said multiple elements of said assessment dataset;
d) determining said at least one metric for elements of said at least one baseline dataset;
e) comparing metrics obtained in step c) with metrics] for elements of said at least one baseline dataset obtained in step d) (See Kushnir, ¶ [0070], The processing circuitry 116 may be configured to select the position of each labeled input 212 and unlabeled input 214 according to the features 304-1, 304-2 of each input from a layer of a neural network 106, and/or to evaluate the unlabeled inputs 218 within the feature space 302. The processing circuitry 116 may be configured to determine a first distance 502, for example, as a distance between the unlabeled input 218 of the pool data set 110 and one or more labeled inputs 212 of the training data set 108. Note: Examiner is interpreting the baseline dataset as the labeled inputs 212);
e) determining which elements of said assessment dataset conform to at least one predetermined condition, said at least one predetermined condition being based on results of step e) (See Kushnir, ¶ [0072], As a first such example, the processing circuitry 116 may be configured to identify the refinement subset 602 as the unlabeled inputs 218 that are distant within the feature space 302 from the labeled inputs 212 (for example, unlabeled inputs 218 having a first distance 502 that is above a distance threshold). The processing circuitry 116 may be configured to include such unlabeled inputs 218 in the refinement subset 602, for example, due to substantial dissimilarity between the unlabeled input 218 and the labeled inputs 212. Note: Examiner interprets the distance or dissimilarity as the predetermined condition);
f) autonomously executing at least one predetermined action for said elements of said first dataset that conform to said at least one predetermined condition (See Kushnir, ¶ [0074], In some example embodiments, the processing circuitry 116 may be configured to submit the refinement subset 602 to a labeling process 220 and to receive, in return, a labeled subset 604. See also [FIG. 2], 220 LABELING PROCESS. Note: Examiner is interpreting the submission of the refinement subset to the labeling process as autonomously executing a predetermined action);
wherein said assessment dataset and said at least one baseline dataset comprise visual data (See Kushnir, ¶ [0110], For example, the training data set 108 may include labeled inputs 212 including video sequences with labels 216 that indicate the events illustrated within the video sequence, …, The pool data set 110 may include unlabeled inputs 218 including video sequences without labels 216. Note: Examiner interprets the unlabeled input as the assessment dataset and the labeled input as the baseline dataset);
wherein said assessment dataset comprises visual data for annotation to result in annotated data, said annotated data being for use in machine learning applications (See Kushnir, ¶ [0103], In some example embodiments, processing circuitry 116 may be configured to include the refinement subset 602 in further training 608 of a partially trained neural network 606. In some other example embodiments, processing circuitry 116 may be configured to use the labeled subset 604 to retrain 406 a neural network 106. Note: Examiner interprets annotated data as the refinement subset 602. Examiner interprets processing the labeled refinement subset further to train a machine learning model as using the annotated data in a machine learning application).
However, Kushnir fail to teach c) determining at least one metric for at least one of said multiple elements of said assessment dataset; d) determining said at least one metric for elements of said at least one baseline dataset; e) comparing metrics obtained in step c) with metrics.
Hazard, working in the same field of endeavor, teaches: c) determining at least one metric for at least one of said multiple elements of said assessment dataset (See Hazard, ¶ [0191], Entropy of a dataset may be a measure of how compressible or predictable a given feature is, which is generally correlated with the skewedness of the class distribution. ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset. Note: Examiner interprets entropy of the synthetic dataset as the metric for the assessment dataset. Examiner interprets the assessment dataset as the synthetic dataset);
d) determining said at least one metric for elements of said at least one baseline dataset (See Hazard, ¶ [0191], Entropy of a dataset may be a measure of how compressible or predictable a given feature is, which is generally correlated with the skewedness of the class distribution. ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset. Note: Examiner interprets entropy of the training dataset as the metric for the baseline dataset. Examiner interprets the baseline dataset as the training dataset).
e) comparing metrics determined in step b) with metrics (See Hazard, ¶ [0192], In some embodiments, a synthetic dataset may be considered more suitable for use (e.g., more likely to be private) when the entropy of the synthetic dataset is higher than the entropy in the training dataset. Note: Examiner interprets entropy as the metric for the assessment dataset. Examiner interprets the assessment dataset as the synthetic dataset. Examiner interprets comparing the entropy as comparing the metrics).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference to have c) determining at least one metric for at least one of said multiple elements of said assessment dataset; d) determining said at least one metric for elements of said at least one baseline dataset; e) comparing metrics obtained in step c) with metrics based on the method of Hazard’s reference to compare two metric of the two datasets. The suggestion/motivation would have been to produce accurate and quality training data for further processing (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 18.
Regarding claim 19, Kushnir discloses a method for assessing a dataset (See Kushnir, [Abstract], for each unlabeled input, a first distance of the unlabeled input to the labeled inputs of the training data set and a second distance of the unlabeled input to other unlabeled inputs of the pool data set. Note: Examiner is interpreting the dataset as the unlabeled inputs from the pool set), the method comprising:
a) receiving said dataset, said dataset having multiple elements (See Kushnir, ¶ [0055], The apparatus 102 stores a pool data set 110 of unlabeled inputs that are not yet associated with a label. Note: the unlabeled inputs of the pool dataset are being interpreted as the dataset) ;
[b) determining at least one metric for at least one of said multiple elements of said dataset;
c) determining at least one predetermined limit for said at least one metric];
d) determining which elements of said dataset conform to at least one predetermined condition, said at least one predetermined condition being based said at least one predetermined limit (See Kushnir, ¶ [0072], As a first such example, the processing circuitry 116 may be configured to identify the refinement subset 602 as the unlabeled inputs 218 that are distant within the feature space 302 from the labeled inputs 212. The processing circuitry 116 may be configured to include such unlabeled inputs 218 in the refinement subset 602, for example, due to substantial dissimilarity between the unlabeled input 218 and the labeled inputs 212. Note: Examiner interprets the distance or dissimilarity as the predetermined condition);
e) autonomously executing at least one predetermined action for said elements of said dataset that conform to said at least one predetermined condition (See Kushnir, ¶ [0074], In some example embodiments, the processing circuitry 116 may be configured to submit the refinement subset 602 to a labeling process 220 and to receive, in return, a labeled subset 604. See also [FIG. 2], 220 LABELING PROCESS. Note: Examiner is interpreting the submission of the refinement subset to the labeling process as autonomously executing a predetermined action);
wherein said dataset comprises visual data for annotation to result in annotated data, said annotated data being for use in machine learning applications (See Kushnir, ¶ [0103], In some example embodiments, processing circuitry 116 may be configured to include the refinement subset 602 in further training 608 of a partially trained neural network 606. In some other example embodiments, processing circuitry 116 may be configured to use the labeled subset 604 to retrain 406 a neural network 106. Note: Examiner interprets annotated data as the refinement subset 602. Examiner interprets processing the labeled refinement subset further to train a machine learning model as using the annotated data in a machine learning application).
However, Kushnir fails to teach b) determining at least one metric for at least one of said multiple elements of said dataset; c) determining at least one predetermined limit for said at least one metric.
Hazard, working in the same field of endeavor, teaches: b) determining at least one metric for at least one of said multiple elements of said dataset (See Hazard, ¶ [0126], The determined 620 set of synthetic data may be assessed in order to calculate various metrics (described herein and may include statistical quality metrics, model comparison metrics, and/or privacy metrics), which are used to determine 630 an overall dataset quality metric. Note: Examiner is interpreting the dataset as the synthetic dataset);
c) determining at least one predetermined limit for said at least one metric (See Hazard, ¶ [0224], In some embodiments, the dataset quality metric is compared 640 to one or more thresholds. Comparing 640 the determined 630 dataset quality metric may entail comparing the dataset quality metric to one or more thresholds. For example, a lower threshold and/or a higher threshold may be used for comparison 640 the dataset quality metric. Note: Examiner interprets the predetermined limit as the threshold);
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s reference to have b) determining at least one metric for at least one of said multiple elements of said dataset; c) determining at least one predetermined limit for said at least one metric based on the method of Hazard’s reference to determine the limit of a metric. The suggestion/motivation would have been to produce accurate and quality training data for further processing for accurate results (See Hazard, ¶ [0003–0006 and 0126]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hazard with Kushnir to obtain the invention as specified in claim 19.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Kushnir et al. (US 20210142168 A1, hereafter, "Kushnir") in view of Hazard et al. (US 20210326652 A1, hereafter, "Hazard") further in view of Zhdanov et al. (US 11048979 B1, hereafter, "Zhdanov").
Regarding claim 3, Kushnir in view of Hazard teaches the method according to claim 1, [wherein said at least one second dataset comprises visual data that is unannotated].
However, Kushnir and Hazard fail to teach wherein said at least one second dataset comprises visual data that is unannotated.
Zhdanov, working in the same field of endeavor, teaches: wherein said at least one second dataset comprises visual data that is unannotated (See Zhdanov, [Col. 5, ln. 19 and 20], ALS 112 can perform active learning for unlabeled or partially unlabeled datasets).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s and Hazard’s reference to wherein said at least one second dataset comprises visual data that is unannotated based on the method of Zhdanov’s reference to use a baseline dataset with labels. The suggestion/motivation would have been to automate annotation and produce accurate annotations using diverse data (See Zhdanov, [Col. 2, ln. 40–50]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Zhdanov with Kushnir and Hazard to obtain the invention as specified in claim 3.
Claims 4, 6, 7, 20 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Kushnir et al. (US 20210142168 A1, hereafter, "Kushnir") in view of Hazard et al. (US 20210326652 A1, hereafter, "Hazard") further in view of Liu et al. (US 20240095880 A1, hereafter, "Liu").
Regarding claim 4, Kushnir in view of Hazard teaches the method according to claim 1, [wherein said metric is a measurement of a characteristic of said visual data, said characteristic being at least one of]:
- [contrast];
- clarity;
- [color levels];
- [black level];
- [white level];
- image quality;
- [color balance];
- [brightness];
- signal-to-noise ratio;
- resolution; and
- number of points in a point cloud.
However, Kushnir and Hazard fail to teach wherein said metric is a measurement of a characteristic of said visual data, said characteristic being at least one of: - contrast; - color levels; - black levels; - white levels; - color balance; - brightness.
Liu, working in the same field of endeavor, teaches: wherein said metric is a measurement of a characteristic of said visual data, said characteristic being at least one of (Note: Superguide and Plain Meaning of "at least of one of x, y, and z" means including all of the elements. Based on the applicants Specification ¶ [0038], “Regarding the set of criteria to be used to determine if one or more datasets or one or more elements of datasets are outliers, this set of criteria can be a set of criteria with a single element (e.g. brightness of an image) or it can be a set of criteria with multiple elements (e.g. contrast of an image, number of pixels in the image, image depth of the image, text associated with the image, and content of the image)”. The metrics do not need to include all of the limitations. So for the purpose of examination the Examiner interprets the characteristics as needing at least one of the limitations and not all of them): - contrast; - color levels; - black levels; - white levels; - color balance; - brightness (See Liu, ¶ [0572], In at least one embodiment, a frame of output frames 5308 is compared with a frame of reference frames 5310 by comparing pixels of both frames with each other. In at least one embodiment, frames are compared by comparing pixel characteristics of frames (e.g., pixel intensity, pixel brightness, pixel color, pixel contrast) and measuring differences in pixel characteristics (e.g., differences in pixel intensity, pixel brightness, pixel color, pixel contrast between pixels of frames)).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s and Hazard’s reference to wherein said metric is a measurement of a characteristic of said visual data, said characteristic being at least one of: - contrast; - color levels; - black levels; - white levels; - color balance; - brightness based on the method of Liu’s reference to represent the metric as a measurement of contrast, color levels, color balance and brightness. The suggestion/motivation would have been to accurately determine the difference between data to determine deviations (See Liu, ¶ [0572]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Liu with Kushnir and Hazard to obtain the invention as specified in claim 4.
Regarding claim 6, Kushnir in view of Hazard teaches the method according to claim 1, [wherein, prior to step b), said elements of said first dataset are pre-processed].
However, Kushnir and Hazard fail to teach wherein, prior to step b), said elements of said first dataset are pre-processed.
Liu, working in the same field of endeavor, teaches: wherein, prior to step b), said elements of said first dataset are pre-processed (See Liu, ¶ [0459], In at least one embodiment, access to DICOM, RIS, CIS, REST compliant, RPC, raw, and/or other data type libraries may be accumulated and pre-processed, including decoding, extracting, and/or performing any convolutions, color corrections, sharpness, gamma, and/or other augmentations to data).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s and Hazard’s reference to wherein, prior to step b), said elements of said first dataset are pre-processed based on the method of Liu’s reference to preprocess the dataset. The suggestion/motivation would have been to accurately determine the difference between data to determine deviations (See Liu, ¶ [0572]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Liu with Kushnir and Hazard to obtain the invention as specified in claim 6.
Regarding claim 7, Kushnir in view of Hazard teaches the method according to claim 6, [wherein pre-processing of said first dataset adjusts a characteristic of said elements].
However, Kushnir and Hazard fail to teach wherein pre-processing of said first dataset adjusts a characteristic of said elements.
Liu, working in the same field of endeavor, teaches: wherein pre-processing of said first dataset adjusts a characteristic of said elements (See Liu, ¶ [0459], In at least one embodiment, access to DICOM, RIS, CIS, REST compliant, RPC, raw, and/or other data type libraries may be accumulated and pre-processed, including decoding, extracting, and/or performing any convolutions, color corrections, sharpness, gamma, and/or other augmentations to data).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s and Hazard’s reference wherein pre-processing of said first dataset adjusts a characteristic of said elements based on the method of Liu’s reference to preprocess the dataset by adjusting the characteristics. The suggestion/motivation would have been to accurately determine the difference between data to determine deviations (See Liu, ¶ [0572]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Liu with Kushnir and Hazard to obtain the invention as specified in claim 7.
Regarding claim 20, claim 20 is rejected the same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to the claim 20, and all of the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Regarding claim 21, claim 21 is rejected the same as claim 7 and the arguments similar to that presented above for claim 7 are equally applicable to the claim 21, and all of the other limitations similar to claim 7 are not repeated herein, but incorporated by reference.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Kushnir et al. (US 20210142168 A1, hereafter, "Kushnir") in view of Hazard et al. (US 20210326652 A1, hereafter, "Hazard") further in view of Zanca et al. (US 20190159747 A1, hereafter, "Zanca").
Regarding claim 16, Kushnir in view of Hazard teaches the method according to claim 12, [wherein said metadata is automatically generated in a pre-processing step applied to one or more datasets].
However, Kushnir and Hazard fail to teach wherein said metadata is automatically generated in a pre-processing step applied to one or more datasets.
Zanca, working in the same field of endeavor, teaches: wherein said metadata is automatically generated in a pre-processing step applied to one or more datasets (See Zanca, ¶ [0020], Accordingly, the imaging system 100, when generating the data set 108, include information in the metadata identifying that the data set 108 corresponds to a test exam (e.g., by tagging the data set as a test exam or as an image not linked to a patient)).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Kushnir’s and Hazard’s reference to wherein said metadata is automatically generated in a pre-processing step applied to one or more datasets based on the method of Zanca’s reference to automatically generate metadata in the preprocessing. The suggestion/motivation would have been to automate the process of generating metadata and address the lack of information (See Zanca, ¶ [0015]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Zanca with Kushnir and Hazard to obtain the invention as specified in claim 16.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chang et al. (US 12430563 B2) teaches systems/techniques that facilitate learning-based clean data selection are provided. In various embodiments, a system can access a raw dataset. In various aspects, the system can select, via execution of a data selection machine learning model, a clean dataset from the raw dataset. In various instances, the system can train a target machine learning model to perform a target task based on the clean dataset. In various aspects, the clean dataset can include candidate-annotation groupings that are in the raw dataset and that are determined by the data selection machine learning model to be suitable for training of the target machine learning model, and the clean dataset can exclude candidate-annotation groupings that are in the raw dataset and that are determined by the data selection machine learning model to not be suitable for training of the target machine learning model.
Wu et al. (US 20190122120 A1) teaches a method and system for augmenting a training dataset for a generative adversarial network (GAN). The training dataset includes labelled data samples and unlabelled data samples. The method includes: receiving generated samples generated using a first neural network of the GAN and the unlabelled samples of training dataset; determining a decision value for a sample from a decision function, wherein the sample is a generated sample of the generated samples or an unlabelled sample of the unlabelled samples of the training dataset; comparing the decision value to a threshold; in response to determining that the decision value exceeds the threshold: predicting a label for a sample; assigning the label to the sample; and augmenting the training dataset to include the sample with the assigned label as a labelled sample.
Badri et al. (US 20230289592 A1) teaches disclosed are methods and systems for generating and using a dataset for training a classifier algorithm. The method comprises inputting a sample dataset into an annotation module; the annotation module ranking a benchmark dataset based on the sample dataset; based on the ranking, the annotation module outputting a subset of the benchmark dataset ranked within a predetermined similarity threshold to the sample dataset; generating a training dataset by adding the subset of the benchmark dataset to the sample dataset; a classification module using the training dataset to train the classifier algorithm. The system comprises a database comprising at least a benchmark dataset; an annotation module configured to receive a sample dataset, rank a benchmark dataset based on the sample dataset; based on the ranking, output a subset of the benchmark dataset ranked within a predetermined similarity threshold to the sample dataset, generate a training dataset by adding the subset of the benchmark dataset to the sample dataset; and a classification module configured to use training dataset to train the classifier algorithm.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DION J SATCHER whose telephone number is (703)756-5849. The examiner can normally be reached Monday - Thursday 5:30 am - 2:30 pm, Friday 5:30 am - 9:30 am PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DION J SATCHER/Patent Examiner, Art Unit 2676
/SHEFALI D GORADIA/Primary Patent Examiner, Art Unit 2676
