Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) was filed on 12/30/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a sample module, used for…” and “a loop module, used for…” in claim 18.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The terms “basic first data”, “basic label”, “basic score” and “basic sample” in claims 12-13 are relative terms which render the claim indefinite. The term “basic first data”, “basic label”, “basic score” and “basic sample” are not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The term “basic first data”, “basic label”, “basic score” and “basic sample” renders the following limitations in claims 12 and 13 as indefinite; From claim 12 “wherein obtaining the plurality of test samples comprises: obtaining basic first data, and setting a basic label and a basic score to form a basic sample; and generating expansive first data by inputting the basic first data into a data generator and setting an expansive label and an expansive score to form an expansive sample” and from claim 13 “wherein setting the expansive score comprises: obtaining a validity of the data generator; and taking a product of the validity and the basic score as the expansive score.”. One of ordinary skill in the art would not be able to determine what are the metes and bounds of a “basic first data”, “basic label”, “basic score” and “basic sample” since it is a subjective term, for example “What is the difference between a basic label and a non-basic label?” “Is it the size, the name, characters, normal?” “A person can see the instructions and interpret them as basic and another person as complex/ non basic.”. Therefore one of ordinary skill in the art would not be able to apprise the scope of the claim for reasons regarding clarity.
The terms “expansive first data”, “expansive label”, “expansive score” and “expansive sample” in claims 12-13 is a relative term which renders the claim indefinite. The terms “expansive first data”, “expansive label”, “expansive score” and “expansive sample” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The terms “expansive first data”, “expansive label”, “expansive score” and “expansive sample” renders the following limitations in claims 12 and 13 as indefinite; From claim 12 “wherein obtaining the plurality of test samples comprises: obtaining basic first data, and setting a basic label and a basic score to form a basic sample; and generating expansive first data by inputting the basic first data into a data generator and setting an expansive label and an expansive score to form an expansive sample” and from claim 13 “wherein setting the expansive score comprises: obtaining a validity of the data generator; and taking a product of the validity and the basic score as the expansive score.”. One of ordinary skill in the art would not be able to determine what are the metes and bounds of an “expansive label”, “expansive score” and “expansive sample” since it is a subjective term, for example “Does expansive in this case mean that it has a capacity to expand or are all directly expanded?” “What is the cause of “expansive” in each case?” “Is expansive more characters?, more labels inside the same label? A higher score? “a bigger sample?” “Is it the dimensions?” Is it simply including other samples inside the same sample? ”. Therefore one of ordinary skill in the art would not be able to apprise the scope of the claim for reasons regarding clarity.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 11-14 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Guo et. al, (US Pub. No. 20220245403 A1) in view of Lee et. al. (US Pub. No. 20200152316 A1) and further in view of PK et. al. (US Pub. No. 20220172108 A1) .
As per claim 11, Guo teaches A test method for an intelligent sensing system, wherein the intelligent sensing system outputs a recognition result about a target based on to-be-recognized data input into the intelligent sensing system, the to-be-recognized data comprises a scenario and the target included in the scenario, and the method comprises:
obtaining a plurality of test samples, wherein each sample comprises a score, a label and first data corresponding to the sample; (See paragraphs 71-77 “[0075] In the embodiments of the present application, by acquiring output data that is outputted by the first intelligent model from processing the input data belonging to the first domain, and training the first intelligent model according to the first sample data and each piece of output data to obtain the second intelligent model, the second intelligent model is applicable to the first domain, thus improving the domain generalization performance of the second intelligent model.” See also paragraphs 82-100 “0083] For each piece of input data inputted to the first intelligent model, the first intelligent model processes the input data and outputs the output data corresponding to the input data. The output data is substantially a processing result obtained by the first intelligent model by processing the input data. The output data includes a confidence value and target box information.” “[0085] Optionally, the output data further includes category information. For example, the category information is a vehicle, a building, a commodity or an animal.” “[0091] In 2031, the first device calculates a value score of each piece of output data according to the output data, wherein each of the value scores is used to indicate a degree of suitability of using a piece of the input data corresponding to a piece of the output data as a piece of the second sample data.” “[0093]… Multiple training samples may be set in advance. Each training sample includes the output data outputted by the intelligent module and a value score obtained by manually labeling the output data. The training sample is fed into the deep learning algorithm and then training of the deep learning algorithm is started. The training process is as follows…”. See also paragraphs 94-124. See also paragraphs 62-66, it shows different scenarios. Guo)
“performing the following steps on each sample:
a. performing scenario presentation on the first data to form to-be-recognized second data; b. inputting the to-be-recognized second data into the intelligent sensing system to obtain a recognition result output from the intelligent sensing system; and” (See paragraphs 88-105 “[0088] In 203, for each piece of output data, the first device sets, according to the output data, annotation information in the input data corresponding to the output data, to obtain second sample data which belongs to the first domain. [0089] The annotation information may be “yes” or “no”, and if the annotation information in the second sample data is “yes”, it indicates that the target image bounded by the target box in the second sample data is a real target. If the annotation information in the second sample data is “no”, it indicates that the target image bounded by the target box in the second sample data is not a real target.” “[0100] Each piece of selected output data includes a confidence value and target box information, and the target box information may include a target box position and a target box size. A target box is determined in the input data corresponding to the output data according to the target box information in the output data. Based on the confidence value, it is determined whether the target bounded by the target box is a real target, and annotation information is set for the target according to the result of determination.” “0101] When the confidence value exceeds a preset confidence threshold, it is determined that the target bounded by the target box in the input data is a real target, and the annotation information set for the target is “yes”. The annotation information “yes” is used to indicate that the target is a real target. When the confidence value does not exceed the preset confidence threshold, it is determined that the target bounded by the target box in the input data is not a real target, and the annotation information set for the target is “no”. The annotation information “no” is used to indicate that the target is not a real target.” See also paragraphs 116-121.) ( “[0117] The first intelligent model uses both the first sample data and the second sample data as sample data. For each piece of sample data, the first intelligent model detects a target in the sample data according to the sample data, compares the detected target with the annotation information in the sample data to obtain difference information, and adjusts parameters of the first intelligent model according to the difference information. The above training process is repeated to obtain the second intelligent model.” Guo), however Guo does not completely teach “c. comparing the recognition result with the label to determine whether the recognition result is correct: in case that the recognition result is correct, taking the score as a first grade corresponding to a current sample from the intelligent sensing system; or in case that the recognition result is incorrect, taking zero as a first grade corresponding to a current sample from the intelligent sensing system; and “dividing a sum of first grades corresponding to all samples by a sum of scores corresponding to all samples to obtain a second grade, wherein the second grade is used to evaluate a sensing capability of the intelligent sensing system.”
Lee teaches “c. comparing the recognition result with the label to determine whether the recognition result is correct: in case that the recognition result is correct, taking the score as a first grade corresponding to a current sample from the intelligent sensing system; or in case that the recognition result is incorrect, taking zero as a first grade corresponding to a current sample from the intelligent sensing system; and” (See paragraphs 24,25, 101, 110 and 140-142 [0095] In some embodiments, the validation may be performed based on an output of a machine learning model. Specifically, when a first annotation result data is acquired from the annotator assigned to the job, the first annotation result data may be compared with a result obtained by inputting a patch of the annotation job to the machine learning model. As a result of the comparison, if it is determined that a difference between the two results exceeds a reference value, the first annotation result data may be suspended or disapproved. “” “[0141] However, as shown in FIG. 23, when the candidate patches 111-1 to 111-n are input, in a case that the calculation model 113 has been learned to output the confidence scores 115-1 to 115-n for each of correct classes and incorrect classes (that is, in a case that the calculation model has been learned to tag label “1” when the prediction matches the correct answer, and otherwise to tag label “0”), the confidence scores of the incorrect classes (shown as underlined) may be used as the misprediction probability. [0142] When the misprediction probability of each of the candidate patches is calculated, the management apparatus 100 may select as the annotation target a candidate patch, having a calculated misprediction probability equal to or greater than a reference value, from the plurality of candidate patches. A high misprediction probability means that the prediction results of the machine learning model are likely to be incorrect since the corresponding patches are important data for improving the performance of the machine learning model. Thus, when patches are selected based on the misprediction probability, high-quality training datasets may be generated because the patches effective for learning are selected as the annotation targets.” Lee)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Guo with the teachings of Lee to compare results with a label to assign a value based on the result when it is correct or incorrect. The modification would have been motivated by the desire to generate high quality datasets, improve learning of the machine learning model, improve performance and ensuring better accuracy, therefore it is an improvement, as suggested by Lee (See parargraphs 140-142, 110, 101 and 31 “[0142]… A high misprediction probability means that the prediction results of the machine learning model are likely to be incorrect since the corresponding patches are important data for improving the performance of the machine learning model. Thus, when patches are selected based on the misprediction probability, high-quality training datasets may be generated because the patches effective for learning are selected as the annotation targets.” “[0101] Furthermore, the accuracy of the annotation results can be ensured by comparing and validating the results of the annotation job with those of the machine learning model or those of other annotators. Accordingly, the performance of the machine learning model that learns the annotation results can also be improved.” Lee)
PK teaches “dividing a sum of first grades corresponding to all samples by a sum of scores corresponding to all samples to obtain a second grade, wherein the second grade is used to evaluate a sensing capability of the intelligent sensing system.” (See paragraphs 49-52 and 60-69. “[0050] At 320, it is determined whether the accuracy score for the machine learning model based on the combined training data block is less than a combined accuracy score that is based on accuracy scores for machine learning models associated with the component training data blocks of the combined training data block. The combined accuracy score for the component training data blocks can be generated, at least in part, by combining accuracy scores for machine learning models generated using the component training data blocks. In a particular embodiment, the combined accuracy score is generated by dividing a sum of the accuracy scores associated with the component data blocks by the number of the component data blocks.” See also paragraphs 31-32. PK)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Guo with the teachings of Lee and PK to divide a sum of all the scores to obtain a second grade to evaluate a sensing capability. The modification would have been motivated by the desire to have better accuracy and avoid poor quality training of the model, therefore it is an improvement, as suggested by PK (See parargraphs 15-20 and 54-55 “0020] By repeating this process, in at least some scenarios, the iterative machine learning and relearning can backtrack to avoid poor quality training records and converge on a training dataset that produces the most accurate machine learning model for a given training dataset.” PK)
Claim 18 is rejected under the same analysis as claim 1.
As per claim 12, Guo in view of Lee and PK teaches “The test method of claim 11, wherein obtaining the plurality of test samples comprises: obtaining basic first data, and setting a basic label and a basic score to form a basic sample; and generating expansive first data by inputting the basic first data into a data generator and setting an expansive label and an expansive score to form an expansive sample.” (The sample presented in the reference can be converted by a set version number, see paragraphs 110-118 “[0113] If the corresponding first sample data is not found in the correspondence, an error prompt may be provided. Alternatively, a version number may be determined, which is lower than the version number of the first intelligent model, and the corresponding first sample data is queried from the correspondence according to the determined version number. The queried first sample data is inputted to a conversion model, which is used to convert the inputted first sample data into the first sample data corresponding to the version number of the first intelligent model. The first sample data corresponding to the version number of the first intelligent model outputted by the conversion model is obtained, and the second intelligent model is obtained by training the first intelligent model based on the obtained first sample data and the second sample data…
[0115] The conversion model is obtained by training a deep learning algorithm with training data in advance. The training data includes sample data corresponding to a first version number and sample data corresponding to a second version number, the first version number being smaller than the second version number. The training data is fed to the deep learning algorithm. The deep learning algorithm converts the sample data corresponding to the first version number to obtain the sample data corresponding to the second version number. The deep learning algorithm adjusts parameters thereof according to the converted sample data corresponding to the second version number and the inputted sample data corresponding to the second version number. By repeating the foregoing process, the deep learning algorithm continuously adjusts the parameters thereof. The deep learning algorithm obtained after the training is stopped is the conversion model.” Guo)
As per claim 13, Guo in view of Lee and PK teaches “The test method of claim 12, wherein setting the expansive score comprises: obtaining a validity of the data generator; and taking a product of the validity and the basic score as the expansive score.” (Examiner interprets “validity” already as a score. See paragraphs 81-88, “0082] In 202, the first device inputs the multiple pieces of input data to the first intelligent model and obtains output data corresponding to each piece of input data, wherein the output data is outputted by the first intelligent model after processing each piece of input data, and includes at least a confidence value and target box information.” See paragraphs 89-100 “[0091] In 2031, the first device calculates a value score of each piece of output data according to the output data, wherein each of the value scores is used to indicate a degree of suitability of using a piece of the input data corresponding to a piece of the output data as a piece of the second sample data.” The score has already been expanded to include the corresponding piece of the second sample data. “[0096] In 2032, the first device selects, according to each of the value scores of the output data, output data meeting a preset condition from all the output data. [0097] The first device may select, from all the output data, output data with value scores exceeding a predetermined score threshold, or the first device may select, from all the output data, a predetermined number of pieces of output data with highest value scores.” The score is also used for the confidence value seen on paragraphs 100-108. Guo)
As per claim 14, Guo in view of Lee and PK teaches “The test method of claim 11, wherein different scores are set for the plurality of test samples based on type differences of the samples.” (See paragraphs 90-100. [0091] In 2031, the first device calculates a value score of each piece of output data according to the output data, wherein each of the value scores is used to indicate a degree of suitability of using a piece of the input data corresponding to a piece of the output data as a piece of the second sample data… [0094] The deep learning algorithm calculates a value score of the output data according to the output data, compares the calculated value score with the value score manually labeled for the output data, to obtain a score difference, and adjusts parameters of the deep learning algorithm based on the score difference. The above training process is repeated to obtain the intelligent analysis model. “0097] The first device may select, from all the output data, output data with value scores exceeding a predetermined score threshold, or the first device may select, from all the output data, a predetermined number of pieces of output data with highest value scores.
[0098] Optionally, the output data may also include data categories, and the output data may be categorized according to the data category included in each piece of output data. For any data category, a predetermined number of pieces of output data satisfying a predetermined condition are selected from all the output data corresponding to the data category based on each of the value scores of the output data. In this way, the output data of each data category can be selected in a balanced manner.” Guo)
As per claim 16, Guo in view of Lee and PK teaches “The test method of claim 11, further comprising: for a first batch of test samples, obtaining a second grade corresponding to the first batch; adding a test sample to the first batch of test samples to form a second batch of test samples; for a second batch of test samples, obtaining a second grade corresponding to the second batch;” (The rejection of claim 11 covers the rejection of claim 16. A second grade of the first batch is already acquired, each piece of output data in considered as the added test sample such as the second data sample. See paragraphs 120-126 “[0122] In the second instance described above, the first sample data in the daytime domain, the output data corresponding to the second sample data in the dark domain, and the second sample data in the dark domain are inputted to the first intelligent model, such that the first intelligent model is trained according to the second sample data in the dark domain, the output data corresponding to the second sample data in the dark domain, and the first sample data in the daytime domain, to obtain the second intelligent model. The second intelligent model obtained through training improves the accuracy of detecting vehicle images in the dark domain.”. See paragraphs 71-77 “[0075] In the embodiments of the present application, by acquiring output data that is outputted by the first intelligent model from processing the input data belonging to the first domain, and training the first intelligent model according to the first sample data and each piece of output data to obtain the second intelligent model, the second intelligent model is applicable to the first domain, thus improving the domain generalization performance of the second intelligent model.” See also paragraphs 82-100. “[0091] In 2031, the first device calculates a value score of each piece of output data according to the output data, wherein each of the value scores is used to indicate a degree of suitability of using a piece of the input data corresponding to a piece of the output data as a piece of the second sample data.” “[0093]… Multiple training samples may be set in advance. Each training sample includes the output data outputted by the intelligent module and a value score obtained by manually labeling the output data. The training sample is fed into the deep learning algorithm and then training of the deep learning algorithm is started. The training process is as follows…”. See also paragraphs 94-124. Guo)) “and in case that the second grade corresponding to the second batch is equal to the second grade corresponding to the first batch, evaluating the sensing capability of the intelligent sensing system by using the second grade corresponding to the second batch; or in case that the second grade corresponding to the second batch is not equal to the second grade corresponding to the first batch, determining that scoring of the sensing capability of the intelligent sensing system is unstable, and continuously adding a test sample for re-scoring.” (The reference PK is already obvious under the main reference Guo, and is also motivated by the same reasons. PK continuously adds test samples (training data blocks) to a sample to obtain another accuracy. See paragraphs 31-32 “[0031] The process of combining training data blocks, generating new machine learning models based on the combined training data blocks, and deactivating the component training data blocks and their associated machine learning models, as described above, can be repeated iteratively. For example, another training data block (e.g., 165) of the plurality of training data blocks 161-167 can be selected and combined with the new training data block 168 to create another new training data block (not shown). In at least some embodiments, the training data block (e.g., 165) can be selected based on an accuracy score associated with the training data block (e.g., 165). The selection can be made from the training data blocks 161-167 that are currently active… Additionally or alternatively, two training data blocks in the training dataset (e.g., 165 and 167) can be combined and used to create a new machine learning model, instead of using the new training data block 168. In at least some such embodiments, the new training data block 168 can be treated like the active training data blocks in the training dataset 130 (e.g., 165 and 167), and two active training data blocks can be selected from the pool of all active training data blocks based on accuracy scores associated with the training data blocks. For example, an accuracy score can be generated for the new machine learning model 158 using the test dataset 140 and used as a selection criterion for the new training data block 168. The iterative process can then repeat again with the remaining active training data blocks, etc.” See also paragraphs 50-57 “[0054] However, if the accuracy score for the combined training data block is greater than or equal to the combined accuracy score for the component training data blocks, then, at 350, the combined training data block and the machine learning model generated using the combined training data block are kept active…[0055] Thus, in at least some scenarios, machine learning models based on combined training data blocks can be unlearned when they are less accurate than the machine learning models based on the component training data blocks of the combined training data block. The component training data blocks, and their associated machine learning models, can be relearned and re-used for subsequent iterations of an iterative machine learning process. However, if the machine learning model generated using the combined training data block is more accurate than, or as accurate as, the machine learning models generated using the component training data blocks, then the combined training data block, and the machine learning model generated using the combined training data block, can be retained and the component training data blocks, and their associated machine learning models, can be forgotten.” See also paragraphs 65-66. PK)
As per claim 17, Guo in view of Lee and PK teaches “The test method of claim 11, wherein the first data comprises a scenario- based real image; and wherein performing the scenario presentation comprises: obtaining all physical objects in the scenario, combining all physical objects based on the real image, and providing combined physical objects to the intelligent sensing system for identification; and/or providing the real image directly to the intelligent sensing system for identification; and/or obtaining part of the physical objects in the scenario, supplementing remaining physical objects through a virtual reality technology, and providing supplemented physical objects to the intelligent sensing system for identification.” (See paragraphs 78-90. The cameras receive image data of vehicles and merges it with captured image data of vehicles. Therefore all are pieces are merged to form multiple pieces of input data. These are then provided to the system for identification, each piece of data corresponds to targets/objects. It presents real images since the images are real. Therefore at least one of the additional limitations is shown by the current reference. [0080] The input data may be, for example, image data. For example, in the first instance, the first intelligent model to be upgraded is a vehicle detection model for detecting vehicle images, the first intelligent model is an intelligent model trained in 2018, and the first device installed with the vehicle detection model is a first camera, that is, the input data belonging to the second domain is images captured by the first camera in 2018. The vehicle detection model continues to be used in 2020 to detect vehicle images of vehicles that have appeared on the market by 2020, that is, the input data belonging to the first domain is images captured by the first camera in 2020. The first camera captures image data that includes images of vehicles have appeared on the market by 2020. In this case, the image data captured by the first camera is the input data belonging to the first domain. Assuming that there is a second camera installed with a vehicle detection model to be upgraded, the second camera captures image data including images of vehicles appearing on the market in 2020, and then sends the captured image data to the first camera. The first camera receives the image data and merges the received image data with captured image data to form multiple pieces of input data belonging to the first domain… [0082] In 202, the first device inputs the multiple pieces of input data to the first intelligent model and obtains output data corresponding to each piece of input data, wherein the output data is outputted by the first intelligent model after processing each piece of input data, and includes at least a confidence value and target box information. [0083] For each piece of input data inputted to the first intelligent model, the first intelligent model processes the input data and outputs the output data corresponding to the input data. The output data is substantially a processing result obtained by the first intelligent model by processing the input data. The output data includes a confidence value and target box information. [0084] Optionally, the target box information may include a target box position and a target box size, and the output data may also include at least one feature such as a data category, a high-level semantic feature, time, a point position, or a description. [0085] Optionally, the output data further includes category information. For example, the category information is a vehicle, a building, a commodity or an animal. “0088] In 203, for each piece of output data, the first device sets, according to the output data, annotation information in the input data corresponding to the output data, to obtain second sample data which belongs to the first domain.” “[0089] The annotation information may be “yes” or “no”, and if the annotation information in the second sample data is “yes”, it indicates that the target image bounded by the target box in the second sample data is a real target. If the annotation information in the second sample data is “no”, it indicates that the target image bounded by the target box in the second sample data is not a real target.”. See also paragraphs 62-66 “0062] An intelligent model is obtained by training a machine learning algorithm. The machine learning algorithm may be a deep learning algorithm or the like, such as a convolutional neural network. For example, the intelligent model may be at least one of a vehicle detection model or an object detection model, etc.” Guo)
As per claim 19, Guo in view of Lee and PK teaches “An electronic device, comprising a memory, a processor and a computer program stored in the memory and executable in the processor, wherein the processor, when executing the computer program, performs steps of a test method for an intelligent sensing system of claim 11.” (See paragraphs 45-49 and 150-156. Guo)
As per claim 20, Guo in view of Lee and PK teaches “A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs steps of a test method for an intelligent sensing system of claim 11.” (See paragraphs 45-49 and 150-156. Guo)
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Lee and PK and further in view of Yi et. al. (CN Pub. No. 108319938 A) .
As per claim 15, Guo in view of Lee and PK teaches “The test method of claim 11, further comprising: classifying the samples based on scores corresponding to the samples; and for samples of different categories, evaluating sensing capabilities of the intelligent sensing system for samples under different categories…” (See paragraphs 93-101 “[0097] The first device may select, from all the output data, output data with value scores exceeding a predetermined score threshold, or the first device may select, from all the output data, a predetermined number of pieces of output data with highest value scores.” “[0098] Optionally, the output data may also include data categories, and the output data may be categorized according to the data category included in each piece of output data. For any data category, a predetermined number of pieces of output data satisfying a predetermined condition are selected from all the output data corresponding to the data category based on each of the value scores of the output data. In this way, the output data of each data category can be selected in a balanced manner.” Guo), however Guo in view of Lee and PK does not teach “in an ascending order of the scores.”
Yi teaches “in an ascending order of the scores.” (See page 20 paragraph 3 “Next, the process identifies the highest similarity score in all similarity scores in the group of images (step 1006). before identifying the highest similarity score, the step to order the calculated similarity score in ascending order or descending order, so as to be good for identifying the highest similarity score…” Yi)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Guo with the teachings of Lee, PK and Yi to order scores as ascending. The modification would have been motivated by the desire to have better identification of the highest scores, therefore it is an improvement, as suggested by Lee (See page 20 paragraph 3 “Next, the process identifies the highest similarity score in all similarity scores in the group of images (step 1006). before identifying the highest similarity score, the step to order the calculated similarity score in ascending order or descending order, so as to be good for identifying the highest similarity score…” Yi)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DYLAN J MENDEZ MUNIZ whose telephone number is (703)756-5672. The examiner can normally be reached M-F, 8AM - 5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DYLAN JOHN MENDEZ MUNIZ/Examiner, Art Unit 2675
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights