DETAILED ACTION
This action is in response to the application filed 07/21/2022. Claims 1-17 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
On page 3, paragraph 0009, “may cause a large difference in performance the model depending on the amount and quality of learning data used in learning” should read “may cause a large different in the performance of the model depending on the amount and quality of learning data used in learning”.
On page 7, paragraph 0025, “G: y of GB information (x, y, z) of pixel information, B: z of GB information (x, y, z) of pixel information” should read “G: y of RGB information (x, y, z) of pixel information, B: z of RGB information (x, y, z) of pixel information”.
On page 23, paragraph 0084, “s pace” should read “space”.
On page 28, paragraph 00107, “G: y of GB information (x, y, z) of pixel information, B: z of GB information (x, y, z) of pixel information” should read “G: y of RGB information (x, y, z) of pixel information, B: z of RGB information (x, y, z) of pixel information”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "labeling a first class specifying the first object image in the bounding box" in lines 12-13. It is unclear as to whether the invention is labeling a first class or if the invention is labeling the first object image with the first class. For examination purposes, Examiner has interpreted this to be labeling a first object image with the first class.
Claim 1 recites the limitation "generating a model for determining a class" in lines 19-20. It is unclear as to whether this model is the same model as previously recited in line 14. For examination purposes, Examiner has interpreted this to be the same model.
Claim 1 recites the limitation "the primarily learned model" in lines 21-22. There is insufficient antecedent basis for this limitation in the claim. It is unclear if this is referring to the generated model or if this is referring to primarily learning a weight of a model. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 1 recites the limitation "by the model" in line 23. There is insufficient antecedent basis for this limitation in the claim. It is unclear which model this is referring to. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 1 recites the limitation "labeling the bounding box specifying a second object image" in lines 22-23. There is insufficient antecedent basis for this limitation in the claim. The bounding box was previously used with the first object image. Therefore, it is unclear as to whether this is a new bounding box or the same bounding box. For examination purposes, Examiner interprets this to be a new bounding box.
Claim 1 recites the limitation "labeling …a second class determined with respect to the second object image by the model" in lines 23-24. It is unclear as to whether the invention is labeling a second class or if the invention is labeling the second object image with the second class. For examination purposes, Examiner has interpreted this to be labeling a second object image with the second class.
Claim 1 recites the limitation "by the model" in line 24. There is insufficient antecedent basis for this limitation in the claim. It is unclear which model this is referring to. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 1 recites the limitation "generating a model for secondarily learning the weight of the model based on the second space image" in lines 26-27. It is unclear as to whether this model is the same model as a previously recited model or if this is referring to a new model. For examination purposes, Examiner has interpreted this to be the same model.
Regarding claims 2-15, claims 2-15 are rejected for at least the same reasons as claim 1 since claims 2-15 depend on claim 1.
Claim 2 recites the limitation "labeling the first class to the bounding box" in lines 7-8. It is unclear as to whether the invention is labeling the first class or if the invention is labeling the bounding box. Additionally, claim 1 claimed “labeling a first class specifying the first object image in the bounding box.” It is unclear as to whether this invention is labeling the first class, labeling the first object image, or labeling the bounding box. For examination purposes, Examiner has interpreted this to be labeling a first object image with the first class.
Claim 3 recites the limitation "secondarily learning a weight of a model" in line 3-4. It is unclear as to whether this model is the same model as a previously recited model or if this is referring to a new model. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 3 recites the limitation "generating a model for determining a class" in line 8. It is unclear as to whether this model is the same model as a previously recited model or if this is referring to a new model. For examination purposes, Examiner has interpreted this to be the same model.
Claim 4 recites the limitation "labeling of the second space image" in line 2. There is insufficient antecedent basis for this limitation in the claim. Claim 1 recites “labeling …a second class determined with respect to the second object image by the model.” It is unclear as to what is being labeled. For examination purposes, Examiner has interpreted this to be labeling a second object image with the second class.
Claim 4 recites the limitation "the primarily learned model" in lines 3-4. There is insufficient antecedent basis for this limitation in the claim. It is unclear if this is referring to the generated model or if this is referring to primarily learning a weight of a model. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 4 recites the limitation "by the model" in line 5. There is insufficient antecedent basis for this limitation in the claim. It is unclear which model this is referring to. For examination purposes, Examiner interprets this to be referring to the generated model.
Regarding claim 6, The term “a greater element value” in line 5 is a relative term which renders the claim indefinite. The term “greater” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The element value has been rendered indefinite by the term “greater”.
Regarding claim 6, The term “a smaller element value” in lines 6-7 is a relative term which renders the claim indefinite. The term “smaller” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The element value has been rendered indefinite by the term “smaller”.
Claim 6 recites the limitation "an element value (x,y,z)" in line 7. It is unclear if this is the same as a previously recited element value or if it’s new element value. Additionally, it is unclear as to whether each element value comprises (x,y,z) RGB information or if it is only for this particular instance of element value. For examination purposes, Examiner interprets this to be describing each previously recited element value.
Claim 7 recites the limitation "pixel information" in line 7. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 7 recites the limitation "pixel information" in line 9. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 8 recites the limitation "pixel information" in line 7. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 8 recites the limitation "GB information (x,y,z)" in line 8. It is unclear if this GB information is the same as RGB information in line 7. For examination purposes, Examiner interprets this to be to be the same.
Claim 8 recites the limitation "pixel information" in line 8. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 8 recites the limitation "GB information (x,y,z)" in line 9. It is unclear if this GB information is the same as RGB information in line 7. For examination purposes, Examiner interprets this to be to be the same.
Claim 8 recites the limitation "pixel information" in line 9. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 8 recites the limitation "pixel information" in line 10. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 9 recites the limitation "pixel information" in line 7. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 9 recites the limitation "pixel information" in line 9. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 9 recites the limitation "R: x’ of (x’, y’, z’) of dst(I) acquired from Equation 4" in line 12. It is unclear how dst(I) is acquired from Equation 4. Therefore, it is unclear if this is the same dst(I) as recited in Equation 3. For examination purposes, Examiner interprets this dst(I) to be acquired from Equation 3.
Claim 9 recites the limitation "G: y’ of (x’, y’, z’) of dst(I) acquired from Equation 4" in line 13. It is unclear how dst(I) is acquired from Equation 4. Therefore, it is unclear if this is the same dst(I) as recited in Equation 3. For examination purposes, Examiner interprets this dst(I) to be acquired from Equation 3.
Claim 9 recites the limitation "B: z’ of (x’, y’, z’) of dst(I) acquired from Equation 4" in line 13-14. It is unclear how dst(I) is acquired from Equation 4. Therefore, it is unclear if this is the same dst(I) as recited in Equation 3. For examination purposes, Examiner interprets this dst(I) to be acquired from Equation 3.
Claim 9 recites the limitation "pixel information" in line 15. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 10 recites the limitation "pixel information" in line 15. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 11 recites the limitation "pixel information" in line 4. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 11 recites the limitation "pixel information" in line 8. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 11 recites the limitation "pixel information" in line 10. It is unclear if this is the same pixel information as recited in claim 1 or if it’s new pixel information. For examination purposes, Examiner interprets this to be the same pixel information.
Claim 12 recites the limitation "pixels" in line 10. It is unclear if this is the same the plurality of pixels or if this is a different set of pixels. For examination purposes, Examiner interprets this to be the same set of pixels.
Regarding claim 13, where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). The term “a standard Gaussian normal distribution” in claim 13 is used by the claim to mean “a standard Gaussian normal distribution with…a standard deviation of 100,” while the accepted meaning is “a standard normal distribution with a standard deviation of 1.” The term is indefinite because the specification does not clearly redefine the term.
Claim 13 recites the limitation "as much as a number of all pixels included in the first space image" in line 5-6. It is unclear as to which value is as much as a number of all pixels when the average value is 0 and the standard deviation is 100. For examination purposes, Examiner interprets this to mean that the amount of pixels in the space image must be greater than the standard deviation.
Claim 13 recites the limitation "to each of the all pixels" in line 8. It is unclear how whether this means “to each of the pixels”, “to all pixels”, or a different interpretation. For examination purposes, Examiner interprets this to mean “to each of the pixels”.
Claim 14 recites the limitation "generating the model" in line 23. There is insufficient antecedent basis for this limitation in the claim. It is unclear which model this is referring to. For examination purposes, Examiner interprets this to be referring to the generated model of claim 1.
Claim 14 recites the limitation "a neural network" in line 8. It is unclear as to whether this is the same neural network as claimed in line 4, or if this is a different neural network. For examination purposes, Examiner interprets this to be the same neural network.
Claim 14 recites the limitation "the input space image" in line 10. There is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether this is referring to the space image set to an input layer in line 3 or if this is referring to another input space image. For examination purposes, Examiner interprets this to be the space image set to an input layer.
Claim 14 recites the limitation "a class of the object" in line 11. It is unclear as to whether this is referring to the same class in line 7 or if this is referring to another class. For examination purposes, Examiner interprets this to be the same class as in line 7.
Claim 16 recites the limitation "labeling a first class specifying the first object image in the bounding box" in lines 7-8. It is unclear as to whether the invention is labeling a first class or if the invention is labeling the first object image with the first class. For examination purposes, Examiner has interpreted this to be labeling a first object image with the first class.
Claim 16 recites the limitation "generating a model for determining a class" in lines 14-15. It is unclear as to whether this model is the same model as previously recited in line 9. For examination purposes, Examiner has interpreted this to be the same model.
Claim 16 recites the limitation "the primarily learned model" in lines 16-17. There is insufficient antecedent basis for this limitation in the claim. It is unclear if this is referring to the generated model or if this is referring to primarily learning a weight of a model. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 16 recites the limitation "by the model" in line 18. There is insufficient antecedent basis for this limitation in the claim. It is unclear which model this is referring to. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 16 recites the limitation "labeling the bounding box specifying a second object image" in lines 17-18. There is insufficient antecedent basis for this limitation in the claim. The bounding box was previously used with the first object image. Therefore, it is unclear as to whether this is a new bounding box or the same bounding box. For examination purposes, Examiner interprets this to be a new bounding box.
Claim 16 recites the limitation "labeling …a second class determined with respect to the second object image by the model" in lines 17-19. It is unclear as to whether the invention is labeling a second class or if the invention is labeling the second object image with the second class. For examination purposes, Examiner has interpreted this to be labeling a second object image with the second class.
Claim 16 recites the limitation "by the model" in line 19. There is insufficient antecedent basis for this limitation in the claim. It is unclear which model this is referring to. For examination purposes, Examiner interprets this to be referring to the generated model.
Claim 16 recites the limitation "generating a model for secondarily learning the weight of the model based on the second space image" in lines 21-22. It is unclear as to whether this model is the same model as a previously recited model or if this is referring to a new model. For examination purposes, Examiner has interpreted this to be the same model.
Regarding claim 17, claim 17 is rejected for at least the same reasons as claim 16 since claim 17 depends on claim 16.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to a computer readable medium (e.g. see claim 17, line 1), which includes a signal based on the broadest reasonable interpretation (i.e. The ordinary and customary meaning of a computer readable medium that includes signals per se). While the Specification discloses a computer readable memory (see page 37), the Specification is not limiting the computer readable medium to only a non-transitory embodiment. A computer readable medium, or the like, that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C 101 by adding the limitation “non-transitory” to the claim and positively reciting that the computer readable medium is a non-transitory computer readable medium. See also In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Dir. 2007) (transitory embodiments are not directed to statutory subject matter). Examiner notes that if Applicant amends to overcome the signals per se rejection, claim 17 will still be rejected under 35 U.S.C. 101.
Claims 1-17 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites a data augmentation-based object analysis model learning apparatus and is thus a machine, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
Generating a second space image by changing pixel information included in the first space image (This limitation is a mental process as it encompasses a human mentally creating a second image by changing pixel information.)
specifying a bounding box in a region including the first object image in the first space image (This limitation is a mental process as it encompasses a human mentally specifying a bounding box.)
labeling a first class specifying the first object image in the bounding box (This limitation is a mental process as it encompasses a human mentally labeling the first object.)
determining a class (This limitation is a mental process as it encompasses a human mentally determining a class.)
labeling the bounding box specifying a second object image in the second space image by the model and a second class determined with respect to the second object image by the model, to the second space image (This limitation is a mental process as it encompasses a human mentally label the bounding box.)
Therefore, claim 1 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 1 further recites additional elements of
A data augmentation-based object analysis model learning apparatus comprising: one or more memories configured to store instructions for performing a predetermined operation; and one or more processors operatively connected to one or more memories and configured to execute the instructions (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Acquiring a first space image including a first object image (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Primarily learning a weight of a model designed based on a predetermined object detection algorithm, for deriving a correlation between the first object image in the bounding box and the first class (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Inputting the first space image to the model (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Generating a model for determining a class (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Inputting the second space image to the primarily learned model (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Generating a model for secondarily learning the weight of the model based on the second space image (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 1 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A data augmentation-based object analysis model learning apparatus comprising: one or more memories configured to store instructions for performing a predetermined operation; and one or more processors operatively connected to one or more memories and configured to execute the instructions uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Acquiring a first space image including a first object image is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Primarily learning a weight of a model designed based on a predetermined object detection algorithm, for deriving a correlation between the first object image in the bounding box and the first class uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Inputting the first space image to the model is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Generating a model for determining a class uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Inputting the second space image to the primarily learned model is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Generating a model for secondarily learning the weight of the model based on the second space image uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 2 recites
The operation further includes generating a set for storing a plurality of classes specifying object information (This limitation is a mental process as it encompasses a human mentally generating a set for storing a plurality of classes.)
Labeling the first class to the bounding box when a bounding box is specified in a region of the first object image in the first space image (This limitation is a mental process as it encompasses a human mentally labeling the bounding box with the first class.)
Therefore, claim 2 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 2 further recites additional elements of
The labeling includes outputting the set to receive selection of the first class specifying the first object image (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Therefore, claim 2 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 2 do not provide significantly more than the abstract idea itself, taken alone and in combination because
The labeling includes outputting the set to receive selection of the first class specifying the first object image is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Therefore, claim 2 is subject-matter ineligible.
Regarding Claim 3:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 3 recites
Specifying the object image included in a space image based on the correlation (This limitation is a mental process as it encompasses a human mentally specifying the object image.)
Determining a class (This limitation is a mental process as it encompasses a human mentally determining a class.)
Therefore, claim 3 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 3 further recites additional elements of
Wherein the generating the secondarily learned model includes secondarily learning a weight of a model, for deriving a correlation between the second object image and the second class (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
By inputting the second space image to the primarily learned model (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Generating a model for determining a class (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 3 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 3 do not provide significantly more than the abstract idea itself, taken alone and in combination because
Wherein the generating the secondarily learned model includes secondarily learning a weight of a model, for deriving a correlation between the second object image and the second class uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
By inputting the second space image to the primarily learned model is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Generating a model for determining a class uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 3 is subject-matter ineligible.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 4 recites
Comparing a second class determined for the second object image with the first class (This limitation is a mental process as it encompasses a human mentally comparing classes.)
Maintaining a value of the second class when the second class and the first class are equal to each other (This limitation is a mental process as it encompasses a human mentally maintaining a value.)
Correcting the value of the second class to a value equal to the first class when the second class and the first class are different from each other (This limitation is a mental process as it encompasses a human mentally correcting the value of the second class.)
Therefore, claim 4 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 4 further recites additional elements of
Wherein the labeling of the second space image includes inputting the second space image to the primarily learned model (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
By the model (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 4 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 4 do not provide significantly more than the abstract idea itself, taken alone and in combination because
Wherein the labeling of the second space image includes inputting the second space image to the primarily learned model is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
By the model uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 4 is subject-matter ineligible.
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 5 recites
Wherein the bounding box is set to include one object image per one bounding box and to include an entire edge region of the object image in the bounding box (This limitation is a mental process as it encompasses a human mentally putting a bounding box in an image including an edge region.)
Therefore, claim 5 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 5 does not further recite any additional elements. Therefore, claim 5 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 5 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 5 is subject-matter ineligible.
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 6 recites
wherein the generating the second space image includes generating the second space image by changing an element value that is greater than a predetermined reference value to a greater element value and changing an element value smaller than the reference value to a smaller element value with respect to an element value (x, y, z) configuring RGB information of the pixel information included in the first space image (This limitation is a mental process as it encompasses a human mentally changing element values.)
Therefore, claim 6 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 6 does not further recite any additional elements. Therefore, claim 6 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 6 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 6 is subject-matter ineligible.
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 7 recites
wherein the generating the second space image includes generating the second space image from the first space image based on Equation 1 below: [Equation 1] dst(I)=round(max(0,min(α*src(I)- β, 255))) (src(I): element value (x, y, z) before pixel information is changed, α: constant, β: constant, and dst(I): element value (x',y',z') after pixel information is changed) (This limitation is a mathematical concept as it encompasses a mathematical equation.)
Therefore, claim 7 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 7 does not further recite any additional elements. Therefore, claim 7 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 7 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 7 is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 8 recites
wherein the generating the second space image includes generating the second space image from the first space image based on Equation 2 below: [Equation 2] Y= 0.1667*R + 0.5*G + 0.3334*B (R: x of RGB information (x, y, z) of pixel information, G: y of GB information (x, y, z) of pixel information, B: z of GB information (x, y, z) of pixel information, and Y: element value (x', y',z') after pixel information is changed). (This limitation is a mathematical concept as it encompasses a mathematical equation.)
Therefore, claim 8 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 8 does not further recite any additional elements. Therefore, claim 8 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 8 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 8 is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 9 recites
wherein the generating the second space image includes generating the second space image from the first space image based on Equations 3 and 4 below: [Equation 3] dst(I)=round(max(0,min(α*src(I)- β, 255))) (src(I): element value (x, y, z) before pixel information is changed, α: constant, β: constant, and dst(I): element value (x',y',z') after pixel information is changed) [Equation 4] Y= 0.1667*R + 0.5*G + 0.3334*B (R: x of RGB information (x, y, z) of pixel information, G: y of GB information (x, y, z) of pixel information, B: z of GB information (x, y, z) of pixel information, and Y: element value (x', y',z') after pixel information is changed). (This limitation is a mathematical concept as it encompasses a mathematical equation.)
Therefore, claim 9 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 9 does not further recite any additional elements. Therefore, claim 9 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 9 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 9 is subject-matter ineligible.
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 10 recites
wherein the generating the second space image includes generating the second space image by adding noise information to some of the pixel information included in the first space image (This limitation is a mental process as it encompasses a human mentally adding noise information to some of the pixel information.)
Therefore, claim 10 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 10 does not further recite any additional elements. Therefore, claim 10 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 10 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 10 is subject-matter ineligible.
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 11 recites
wherein the generating the second space image includes generating the second space image by adding noise information to pixel information of the first space image based on Equation 5 below: [Equation 5] dst(I)=round(max(0,min(src(I)
±
N, 255))) (src(I): element value (x, y, z) before pixel information is changed, N: random number, dst(I) : element value (x', y', z') after pixel information is changed) (This limitation is a mathematical concept as it encompasses a mathematical equation.)
Therefore, claim 11 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 11 does not further recite any additional elements. Therefore, claim 11 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 11 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 11 is subject-matter ineligible.
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 12 recites
wherein the generating the second space image includes generating the second space image by calculating a value (Rmax-RAVG, Gmax-GAVG, Bmax-BAVG) by subtracting an element average value (RAVG, GAVG, BAVG) of each of R, G, and B of a plurality of pixels from a maximum element value (Rmax, Gmax, Bmax) among element values of each of R, G, and B of the plurality of pixels included in a size of an NxN matrix (N being a natural number equal to or greater than 3) including a first pixel at a center among pixels included in the first space image and, when any one of element values of the (Rmax-RAVG, Gmax-GAVG, Bmax-BAVG) is smaller than a preset value, performing an operation of blurring the first pixel. (This limitation is a mathematical concept as it encompasses a mathematical equation.)
Therefore, claim 12 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 12 does not further recite any additional elements. Therefore, claim 12 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 12 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 12 is subject-matter ineligible.
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 13 recites
wherein the generating the second space image includes generating random number information based on standard Gaussian normal distribution with an average value of 0 and a standard deviation of 100 as much as a number of all pixels included in the first space image and generating the second space image into which noise is inserted by adding the random number information to each of the all pixels (This limitation is a mental process as it encompasses a human mentally generating a random number and adding it to each pixel.)
Therefore, claim 13 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 13 does not further recite any additional elements. Therefore, claim 13 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 13 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 13 is subject-matter ineligible.
Regarding Claim 14:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 14 recites
a correlation for determining a class of the object image included in the input space image (This limitation is a mental process as it encompasses a human mentally creating a correlating between a class and the object image.)
Therefore, claim 14 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements of
wherein the generating the model includes setting a space image including an object image to an input layer of a neural network designed based on a faster region-based convolutional neural network (Faster R-CNN) algorithm, setting a bounding box including the object image and a class of the object image to an output layer (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
learning a weight of a neural network for deriving a correlation of a region of the bounding box of the object image included in the space image, input from the input space image (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 14 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 14 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the generating the model includes setting a space image including an object image to an input layer of a neural network designed based on a faster region-based convolutional neural network (Faster R-CNN) algorithm, setting a bounding box including the object image and a class of the object image to an output layer is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
learning a weight of a neural network for deriving a correlation of a region of the bounding box of the object image included in the space image, input from the input space image uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 14 is subject-matter ineligible.
Regarding Claim 15:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 15 recites the same abstract ideas as claim 1. Therefore, claim 15 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 15 further recites additional elements of
An apparatus including a data augmentation-based object analysis model generated by the apparatus of claim 1 (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 15 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 15 do not provide significantly more than the abstract idea itself, taken alone and in combination because
An apparatus including a data augmentation-based object analysis model generated by the apparatus of claim 1 uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 15 is subject-matter ineligible.
Regarding Claim 16:
Subject Matter Eligibility Analysis Step 1:
Claim 16 recites a method and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 16 recites
Generating a second space image by changing pixel information included in the first space image (This limitation is a mental process as it encompasses a human mentally creating a second image by changing pixel information.)
specifying a bounding box in a region including the first object image in the first space image (This limitation is a mental process as it encompasses a human mentally specifying a bounding box.)
labeling a first class specifying the first object image in the bounding box (This limitation is a mental process as it encompasses a human mentally labeling the first object.)
determining a class (This limitation is a mental process as it encompasses a human mentally determining a class.)
labeling the bounding box specifying a second object image in the second space image by the model and a second class determined with respect to the second object image by the model, to the second space image (This limitation is a mental process as it encompasses a human mentally label the bounding box.)
Therefore, claim 16 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 16 further recites additional elements of
a method performed by a data augmentation-based object analysis learning apparatus (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Acquiring a first space image including a first object image (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Primarily learning a weight of a model designed based on a predetermined object detection algorithm, for deriving a correlation between the first object image in the bounding box and the first class (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Inputting the first space image to the model (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Generating a model for determining a class (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Inputting the second space image to the primarily learned model (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Generating a model for secondarily learning the weight of the model based on the second space image (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 16 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 16 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A method performed by a data augmentation-based object analysis learning apparatus uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Acquiring a first space image including a first object image is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Primarily learning a weight of a model designed based on a predetermined object detection algorithm, for deriving a correlation between the first object image in the bounding box and the first class uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Inputting the first space image to the model is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Generating a model for determining a class uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Inputting the second space image to the primarily learned model is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Generating a model for secondarily learning the weight of the model based on the second space image uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 16 is subject-matter ineligible.
Regarding Claim 17:
Subject Matter Eligibility Analysis Step 1:
Claim 17 recites a computer program recorded in a computer-readable medium and is thus signals per se and not one of the statutory categories. Examiner notes that for the purpose of compact prosecution, claim 17 will be examined below as if it were amended to fall within a statutory category.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 17 recites the same abstract ideas as claim 16. Therefore, claim 17 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 17 further recites additional elements of
A computer program recorded in a computer-readable recording medium for performing the method of claim 16 by a processor (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 17 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 17 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A computer program recorded in a computer-readable recording medium for performing the method of claim 16 by a processor uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 17 is subject-matter ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 5, 10, and 14-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha et al. (US 2021/0150282 A1) (hereafter referred to as Chadha) in view of Huynh et al. (US 11,106,903 B1) (hereafter referred to as Huynh).
Regarding claim 1, Chadha teaches
A data augmentation-based object analysis model learning apparatus comprising: one or more memories configured to store instructions for performing a predetermined operation (Chadha, page 22, paragraph 0127, “An apparatus for image processing is described. The apparatus may include a processor, memory coupled with the processor, and instructions stored in the memory. The instructions may be executable by the processor to cause the apparatus to manipulate an input image to generate at least one modified input image”);
and one or more processors operatively connected to the one or more memories and configured to execute the instructions, wherein the operation performed by the processor includes (Chadha, page 19, paragraph 0092, “The processor 630 may be configured to execute computer-readable instructions stored in a memory 625 to perform various functions (e.g., functions or tasks supporting image augmentation and object detection).”):
acquiring a first space image including a first object image and generating a second space image by changing pixel information included in the first space image (Chadha, page 14, paragraph 0029 “The system supports manipulation of the input image to produce at least one modified input image 210. As illustrated, the system manipulates the input image 205 to generate a first scaled image 210-a and a second scaled image 210-b. The first scaled image 210-a is a 50x50 pixel version of the input image 205, and the second scaled image 210-b is a 200x200 pixel version of the input image 205” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the first space image is the input image and the second space image is the modified input image.);
specifying a bounding box in a region including the first object image in the first space image (Chadha, page 14, paragraph 0031, “The object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects.” )
inputting the second space image to the primarily learned model and …the bounding box specifying a second object image in the second space image (Chadha, page 14, paragraph 0029 “The system supports manipulation of the input image to produce at least one modified input image 210. As illustrated, the system manipulates the input image 205 to generate a first scaled image 210-a and a second scaled image 210-b. The first scaled image 210-a is a 50x50 pixel version of the input image 205, and the second scaled image 210-b is a 200x200 pixel version of the input image 205” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the first space image is the input image and the second space image is the modified input image.)
and generating a model for secondarily learning the weight of the model based on the second space image (Chadha, page 14, paragraph 0030, “The input image 205 and the modified input images 210 may be processed by an object detection machine learning model” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the weight is the confidence scores and the second space image is the modified input images.).
Chadha does not explicitly disclose, but Huynh does disclose
and labeling a first class specifying the first object image in the bounding box (Huynh, page 13, column 2, lines 9-14, “Annotated training data comprises labeled image data including data representing bounding boxes that identify the location of objects represented in the image data and classification data identifying a class of the object (e.g., data representing ‘cat’, ‘dog’, ‘human’, etc.)” );
primarily learning a weight of a model designed based on a predetermined object detection algorithm, for deriving a correlation between the first object image in the bounding box and the first class, by inputting the first space image to the model, specifying an object image included in a space image based on the correlation, and generating a model for determining a class (Huynh, page 20, column 16, lines 13-16, “the detector may output classification data indicating a confidence value that an object depicted within the bounding box is a dog” where “processing may begin at action 1110, ‘Receive input frame of image data’. At action 1110, the detector 118 may receive an input frame of image data. In various examples, the input frame of image data may comprise a plurality of pixel addresses arranged in a two-dimensional grid. Each pixel address may be associated with a pixel value representing a color, brightness, and/or other parameter of the pixel address.” Examiner notes that the weight is the confidence value, the first space image is the input frame of image data, and the model is the detector.);
labeling the bounding box specifying a …object image in the … space image by the model and a … class determined with respect to the … object image by the model, to the … space image (Huynh, page 20, column 16, lines 13-16, “the detector may output classification data indicating a confidence value that an object depicted within the bounding box is a dog” where “processing may begin at action 1110, ‘Receive input frame of image data’. At action 1110, the detector 118 may receive an input frame of image data. In various examples, the input frame of image data may comprise a plurality of pixel addresses arranged in a two-dimensional grid. Each pixel address may be associated with a pixel value representing a color, brightness, and/or other parameter of the pixel address.” Examiner notes that the space image is the input frame of image data, and the model is the detector.);
Chadha and Huynh are considered analogous to the claimed invention because they both detect objects using machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha to label the objects detected in the bounding box like in Huynh. Doing so is advantageous because “large datasets comprising annotated training data (e.g., labeled image data) exist and may be used to train various computer vision models” (Huynh, page 13, column 2, lines 21-22).
Regarding claim 2, Chadha teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha does not teach, but Huynh does teach
the operation further includes generating a set for storing a plurality of classes specifying object information (Huynh, page 14, column 4, lines 11-16, “In various examples, detector 118 may be effective to locate and/or classify various objects depicted in a frame of image data. For example, detector 118 may be effective to locate objects representing cats and dogs within frames of image data and may be effective to classify located objects as either cats, dogs, or neither.” Examiner notes that the set for storing a plurality of classes is the set of classifications of cats and dogs.);
and the labeling includes outputting the set to receive selection of the first class specifying the first object image and labeling the first class to the bounding box when a bounding box is specified in a region of the first object image in the first space image (Huynh, page 14, column 4, lines 11-16, “In various examples, detector 118 may be effective to locate and/or classify various objects depicted in a frame of image data. For example, detector 118 may be effective to locate objects representing cats and dogs within frames of image data and may be effective to classify located objects as either cats, dogs, or neither.” Examiner notes that the set is the set of classifications of cats and dogs and the bounding box is the frames of image data.).
Chadha and Huynh are considered analogous to the claimed invention because they both detect objects using machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha to label the objects detected in the bounding box like in Huynh. Doing so is advantageous because “large datasets comprising annotated training data (e.g., labeled image data) exist and may be used to train various computer vision models” (Huynh, page 13, column 2, lines 21-22).
Regarding claim 3, Chadha teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha further teaches
secondarily learning a weight of a model, for deriving … the second object image …, by inputting the second space image to the primarily learned model (Chadha, page 20, paragraph 0096, “At 715, the image detection system may generate, using the object detection machine learning model, a first range of confidence scores corresponding to the first set of objects and at least one second range of confidence scores corresponding to the at least one second set of objects” where “at 710, the image detection system may identify, using an object detection machine learning model, the input image and the at least one modified input image, a first set of objects from the input image and at least one second set of objects corresponding to the at least one modified input image” (Chadha, page 19-20, paragraph 0095). Examiner notes that the second set of objects is from the second space image, and the weight of the model is confidence scores.)
specifying the object image included in a space image based on the correlation (Chadha, page 14, paragraph 0031, “The object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects.” ),
and generating a model for determining a class (Chadha, page 12, paragraph 0014, “A set of training images may be manipulated and used to train the model such that the model is better equipped to detect or classify images and/or detect objects within images.”).
Chadha does not teach, but Huynh does teach
deriving a correlation between the second object image and the second class, by inputting the second space image to the primarily learned model (Huynh, page 20, column 16, lines 13-16, “the detector may output classification data indicating a confidence value that an object depicted within the bounding box is a dog” where “processing may begin at action 1110, ‘Receive input frame of image data’. At action 1110, the detector 118 may receive an input frame of image data. In various examples, the input frame of image data may comprise a plurality of pixel addresses arranged in a two-dimensional grid. Each pixel address may be associated with a pixel value representing a color, brightness, and/or other parameter of the pixel address.” Examiner notes that the second space image is the input frame of image data, and the model is the detector.)
Chadha and Huynh are considered analogous to the claimed invention because they both detect objects using machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha to label the objects detected in the bounding box like in Huynh. Doing so is advantageous because “large datasets comprising annotated training data (e.g., labeled image data) exist and may be used to train various computer vision models” (Huynh, page 13, column 2, lines 21-22).
Regarding claim 5, Chadha teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha further teaches
wherein the bounding box is set to include one object image per one bounding box and to include an entire edge region of the object image in the bounding box (Chadha, page 15, paragraph 0036, “The object detection neural network 220 may identify bounding boxes that outline detected objects” and “The machine learning model 320 may also generate a range of confidence scores for each image, where each confidence score corresponds to a detected object (and a bounding box for the detected object)” (Chadha, page 16, paragraph 0040). Examiner notes the entire edge region of the object image is the outline of the object.).
Regarding claim 10, Chadha teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha further teaches
wherein the generating the second space image includes generating the second space image by adding noise information to some of pixel information included in the first space image (Chadha, page 14, paragraph 0029, “Other image manipulations may be performed by the system 200 in addition to or instead of scaling. For example, the system may flip images, stretch or squeeze images, add noise to image data, brighten or darken the input image 205, transform a perspective of the input image 205, or a combination of these and other image manipulation techniques.”).
Regarding claim 14, Chadha teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha does not teach, but Huynh does teach
wherein the generating the model includes setting a space image including an object image to an input layer of a neural network designed based on a faster region-based convolutional neural network (Faster R-CNN) algorithm (Huynh, page 14, column 4, lines 30-33, “Detector 118 may receive a frame of input image data 108 that is in the NIR domain. Feature data may be extracted from the input NIR image data at a particular block or level of the detector 118” where “In various examples, deep learning techniques may be used to detect objects in image data. Convolutional neural networks (CNNs), single shot detectors (SSDs), region-convolutional neural networks (R-CNNs), Faster R-CNN, Region based Fully Convolutional Networks (R-FCNs) and other machine learning models may be trained to perform object detection and classification” (Huynh, page 13, column 2, lines 1-7). Examiner notes that the space image is input at a block or an input layer of the detector. ),
setting a bounding box including the object image and a class of the object image to an output layer (Huynh, page 14, column 4, lines 46-50, “The box predictor 134 may be an output layer of detector 118 effective to detect one or more objects represented by the feature data and may output annotated data describing the bounding boxes that locate and/or classify relevant detected objects in the scene.”),
and learning a weight of a neural network for deriving a correlation of a region of the bounding box of the object image included in the space image, input from the input space image, and a correlation for determining a class of the object image included in the input space image (Huynh, page 20, column, 16, lines 5-16, “At action 1150, the detector may output a bounding box corresponding to the particular feature data extracted at action 1120. The bounding box may identify a grouping of contiguous pixels (e.g., by identifying contiguous pixel addresses) within the frame of image data received at action 1110. In various examples, the detector may also classify an object within the bounding box. For example, if the detector is trained to detect dogs, the detector may output classification data indicating a confidence value that an object depicted within the bounding box is a dog” where “processing may continue from action 1110 to action 1120, ‘Extract feature data from input frame of image data’. At action 1120, the detector 118 may extract a feature vector or feature map from the input frame of image data” (Huynh, page 20, column 19, lines, 48-51) Examiner notes that the weight is the confidence value, the input from the input space image is the feature data that is used for generating the bounding box, and the confidence value is the correlation for determining a class of the object image.).
Chadha and Huynh are considered analogous to the claimed invention because they both detect objects using machine learning. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to have implemented Chadha on a Faster R-CNN like in Huynh instead of the model in Chadha. Thus, this would be simple substitution of one known element (machine learning model) for another (Faster R-CNN) to obtain predictable results (detect objects in images) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results). It also would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha to label the objects detected in the bounding box like in Huynh. Doing so is advantageous because “large datasets comprising annotated training data (e.g., labeled image data) exist and may be used to train various computer vision models” (Huynh, page 13, column 2, lines 21-22).
Regarding claim 15, Chadha teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha further teaches
An apparatus including a data augmentation-based object analysis model generated by the apparatus of claim 1 (Chadha, page 14, paragraph 0030, “The input image 205 and the modified input images 210 may be processed by an object detection machine learning model” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the apparatus is the object detection machine learning model.).
Regarding claim 16, Chadha teaches
A method performed by a data augmentation-based object analysis learning apparatus, (Chadha, page 22, paragraph 0127, “An apparatus for image processing is described. The apparatus may include a processor, memory coupled with the processor, and instructions stored in the memory. The instructions may be executable by the processor to cause the apparatus to manipulate an input image to generate at least one modified input image”);
the method comprising: acquiring a first space image including a first object image and generating a second space image by changing pixel information included in the first space image (Chadha, page 14, paragraph 0029 “The system supports manipulation of the input image to produce at least one modified input image 210. As illustrated, the system manipulates the input image 205 to generate a first scaled image 210-a and a second scaled image 210-b. The first scaled image 210-a is a 50x50 pixel version of the input image 205, and the second scaled image 210-b is a 200x200 pixel version of the input image 205” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the first space image is the input image and the second space image is the modified input image.);
specifying a bounding box in a region including the first object image in the first space image (Chadha, page 14, paragraph 0031, “The object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects.” )
inputting the second space image to the primarily learned model and …the bounding box specifying a second object image in the second space image (Chadha, page 14, paragraph 0029 “The system supports manipulation of the input image to produce at least one modified input image 210. As illustrated, the system manipulates the input image 205 to generate a first scaled image 210-a and a second scaled image 210-b. The first scaled image 210-a is a 50x50 pixel version of the input image 205, and the second scaled image 210-b is a 200x200 pixel version of the input image 205” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the first space image is the input image and the second space image is the modified input image.)
and generating a model for secondarily learning the weight of the model based on the second space image (Chadha, page 14, paragraph 0030, “The input image 205 and the modified input images 210 may be processed by an object detection machine learning model” where “the object detection neural network 220 may identify sets of objects (e.g., an object identified by a bounding box 225) for each processed image and confidence scores associated with the identified objects” (Chadha, page 14, paragraph 0031). Examiner notes that the weight is the confidence scores and the second space image is the modified input images.).
Chadha does not explicitly disclose, but Huynh does disclose
and labeling a first class specifying the first object image in the bounding box (Huynh, page 13, column 2, lines 9-14, “Annotated training data comprises labeled image data including data representing bounding boxes that identify the location of objects represented in the image data and classification data identifying a class of the object (e.g., data representing ‘cat’, ‘dog’, ‘human’, etc.)” );
primarily learning a weight of a model designed based on a predetermined object detection algorithm, for deriving a correlation between the first object image in the bounding box and the first class, by inputting the first space image to the model, specifying an object image included in a space image based on the correlation, and generating a model for determining a class (Huynh, page 20, column 16, lines 13-16, “the detector may output classification data indicating a confidence value that an object depicted within the bounding box is a dog” where “processing may begin at action 1110, ‘Receive input frame of image data’. At action 1110, the detector 118 may receive an input frame of image data. In various examples, the input frame of image data may comprise a plurality of pixel addresses arranged in a two-dimensional grid. Each pixel address may be associated with a pixel value representing a color, brightness, and/or other parameter of the pixel address.” Examiner notes that the weight is the confidence value, the first space image is the input frame of image data, and the model is the detector.);
labeling the bounding box specifying a …object image in the … space image by the model and a … class determined with respect to the … object image by the model, to the … space image (Huynh, page 20, column 16, lines 13-16, “the detector may output classification data indicating a confidence value that an object depicted within the bounding box is a dog” where “processing may begin at action 1110, ‘Receive input frame of image data’. At action 1110, the detector 118 may receive an input frame of image data. In various examples, the input frame of image data may comprise a plurality of pixel addresses arranged in a two-dimensional grid. Each pixel address may be associated with a pixel value representing a color, brightness, and/or other parameter of the pixel address.” Examiner notes that the space image is the input frame of image data, and the model is the detector.);
Chadha and Huynh are considered analogous to the claimed invention because they both detect objects using machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha to label the objects detected in the bounding box like in Huynh. Doing so is advantageous because “large datasets comprising annotated training data (e.g., labeled image data) exist and may be used to train various computer vision models” (Huynh, page 13, column 2, lines 21-22).
Regarding claim 17, Chadha teaches the method of claim 16. Chadha further teaches
A computer program recorded in a computer-readable recording medium for performing the method of claim 16 by a processor (Chadha, page 19, paragraph 0092, “The processor 630 may be configured to execute computer-readable instructions stored in a memory 625 to perform various functions (e.g., functions or tasks supporting image augmentation and object detection).”).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha in view of Huynh in further view of Zagaynov et al. (US 2021/0224969 A1) (hereafter referred to as Zagaynov).
Regarding claim 4, Chadha in view of Huynh teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha in view of Huynh further teaches
Wherein labeling of the second space image includes inputting the second space image to the primarily learned model (Chadha, page 20, paragraph 0096, “At 715, the image detection system may generate, using the object detection machine learning model, a first range of confidence scores corresponding to the first set of objects and at least one second range of confidence scores corresponding to the at least one second set of objects” where “at 710, the image detection system may identify, using an object detection machine learning model, the input image and the at least one modified input image, a first set of objects from the input image and at least one second set of objects corresponding to the at least one modified input image” (Chadha, page 19-20, paragraph 0095). Examiner notes that the second set of objects is from the second space image, and the weight of the model is confidence scores.)
Chadha in view of Huynh does not teach, but Zagaynov does teach
comparing a second class determined for the second object image with the first class by the model, maintaining a value of the second class when the second class and the first class are equal to each other, and correcting the value of the second class to a value equal to the first class when the second class and the first class are different from each other (Zagaynov, page 16, paragraph 0042-0043, “The observed output of the neural network OUTPUTNN (TRAINING INPUT) is compared with the desired training output 124 specified by the training data set: [0043] Compare: OUTPUTNN(TRAINING INPUT) vs. TRAINING OUTPUT. The resulting error – the difference between the output of the neural network OUTPUTNN and the desired TRAINING OUTPUT is propagated back to the previous layers of the neural network, in which the weights are adjusted so as to modify the OUTPUTNN and make it closer to the TRAINING OUTPUT. This adjustment may be repeated until the output error for a particular training input satisfies a predetermined condition (e.g., falls below a predetermined error)” where “the neural network may have output channels configured to output indications of the type of object, in addition to identification of the object as belonging to a particular class” (Zagaynov, page 14, paragraph 0027) and “The techniques described herein allow for automatic detection of objects in images using artificial intelligence” (Zagaynov, page 14, paragraph 0026). Examiner notes that the comparing the second class with the first class is comparing the OUTPUTNN with the TRAINING OUTPUT. Examiner further notes that if the classes are different, the OUTPUTNN- is changed to equal the first TRAINING OUTPUT. Examine additionally notes that when the outputs are equal to each other, the values are maintained.).
Chadha, Huynh, and Zagaynov are considered analogous to the claimed invention because they detect objects in machine learning. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to compare classes like in Zagaynov. Doing so is advantageous because “the parameters of the neural network may be adjusted to optimize prediction accuracy” (Zagaynov, page 14, paragraph 0028).
Claim(s) 6-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha in view of Huynh in further view of Mathematics Stack Exchange (“Algorithms to increase or decrease the contrast of an image”) (hereafter referred to as Mathematics Stack Exchange).
Regarding claim 6, Chadha in view of Huynh teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha in view of Huynh does not teach, but Oseiskar does teach
wherein the generating the second space image includes generating the second space image by changing an element value that is greater than a predetermined reference value to a greater element value and changing an element value smaller than the reference value to a smaller element value with respect to an element value (x, y, z) configuring RGB information of the pixel information included in the first space image (Mathematics Stack Exchange, page 2, “The basic contrast and brightness adjustments are transformations of the form f(x) = αx + β (with the result rounded to an integer and clamped to the range [0,225].). Here x is a color component value (R, G, or B). The slope α controls contrast (α >1 means more contrast and 0<α<1 less contrast). For easier separation of ‘brightness’ and ‘contrast’ modifications, the formula can be written like f(x) = α(x -128) + 128 + b where b controls brightness.” Examiner notes that 128 is the predetermined reference value and when a value above 128 is input into the equation, the value changes to a greater value, and when a value below 128 is input into the equation, the value changes to a lesser value).
Chadha, Huynh, and Oseiskar are considered analogous to the claimed invention because they modify images. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to generate the second space image like in Oseiskar. Thus, this would be simple substitution of one known element (generating the second space image like in Chadha in view of Huynh) for another (generating the second space image like in Oseiskar) to obtain predictable results (generate a second space image) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Regarding claim 7, Chadha in view of Huynh and Oseiskar teaches the data augmentation-based object analysis model learning apparatus of claim 6. Chadha in view of Huynh and Oseiskar further teach
wherein the generating the second space image includes generating the second space image from the first space image based on Equation 1 below: [Equation 1] dst(I)=round(max(0,min(α*src(I)- β, 255))) (src(I): element value (x, y, z) before pixel information is changed, α: constant, β: constant, and dst(I): element value (x',y',z') after pixel information is changed) (Mathematics Stack Exchange, page 2, “The basic contrast and brightness adjustments are transformations of the form f(x) = αx + β (with the result rounded to an integer and clamped to the range [0,225].). Here x is a color component value (R, G, or B). The slope α controls contrast (α >1 means more contrast and 0<α<1 less contrast). For easier separation of ‘brightness’ and ‘contrast’ modifications, the formula can be written like f(x) = α(x -128) + 128 + b where b controls brightness.” Examiner notes the color component value is the element value, x is src(I), f(x) is dst(I), α is α, and β is β. Examiner further notes that this equation is rounded to an integer and clamped to the range [0,255].) .
Chadha, Huynh, and Oseiskar are considered analogous to the claimed invention because they modify images. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to generate the second space image like in Oseiskar. Thus, this would be simple substitution of one known element (generating the second space image like in Chadha in view of Huynh) for another (generating the second space image like in Oseiskar) to obtain predictable results (generate a second space image) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha in view of Huynh in further view of Cook (“Three algorithms for converting color to grayscale”) (hereafter referred to as Cook), and Dawei et al. (CN110986766A) (hereafter referred to as Dawei)
Regarding claim 8, Chadha in view of Huynh teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha in view of Huynh does not teach, but Cook does teach an equation to generate the second space image according to a weighted average of the RGB information. Thus, Cook teaches
wherein the generating the second space image includes generating the second space image from the first space image based on Equation 2 below: [Equation 2] Y= 0.1667*R + 0.5*G + 0.3334*B (R: x of RGB information (x, y, z) of pixel information, G: y of GB information (x, y, z) of pixel information, B: z of GB information (x, y, z) of pixel information, and Y: element value (x', y',z') after pixel information is changed) (Cook, page 1, 4th paragraph, “The luminosity method is a more sophisticated version of the average method. It also averages the values, but it forms a weighted average to account for human perception. We’re more sensitive to green than other colors, so green is weighted most heavily. The formula for luminosity is 0.21R +0.72G +0.07B”)
Chadha, Huynh, and Cook are considered analogous to the claimed invention because they modify images. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to generate the second space image like in Cook. Thus, this would be simple substitution of one known element (generating the second space image like in Chadha in view of Huynh) for another (generating the second space image like in Cook) to obtain predictable results (generate a second space image) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Chadha, Huynh, and Cook does not explicitly teach the coefficients of the equation, but Dawei teaches that the coefficients are customizable used in the weighted average to convert an image to grayscale. Thus Dawei teaches
[Equation 2] Y= 0.1667*R + 0.5*G + 0.3334*B (Dawei, page 12, paragraph 0068 “S101. Use a custom weighted average method to convert a three-channel color image into a single-channel grayscale image. Let the pixel values of each pixel in the color image before conversion be R(x,y), G(x,y), and B(x,y), and the grayscale value of each pixel in the converted grayscale image be Gary(x,y). The conversion rule is Gary(x,y)=a x R(x,y) + b x G(x,y) + c x B(x,y), where a+b+c=1. The values of a, b, and c are customizable.”).
Chadha, Huynh, Cook and Dawei are considered analogous to the claimed invention because they use a weighted average. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh and Cook to use the custom coefficients as in Dawei. Thus, this would be simple substitution of one known element (coefficients in Chadha, Huynh, and Cook) for another (coefficients in Dawei) to obtain predictable results (a weighted average in the form of Equation 2) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha in view of Huynh in further view of Oseiskar, Cook, Dawei, and Tweakimp et al. (“How Can I increase the contrast in a color image converted to grayscale?”) (hereafter referred to as Tweakimp).
Regarding claim 9, Chadha in view of Huynh teaches the data augmentation-based object analysis model learning apparatus of claim 1. Chadha in view of Huynh does not teach, but Oseiskar does teach
wherein the generating the second space image includes generating the second space image from the first space image based on Equation[] 3… below: [Equation 3] dst(I)=round(max(0,min(α*src(I)- β, 255))) (src(I): element value (x, y, z) before pixel information is changed, α: constant, β: constant, and dst(I): element value (x',y',z') after pixel information is changed) (Mathematics Stack Exchange, page 2, “The basic contrast and brightness adjustments are transformations of the form f(x) = αx + β (with the result rounded to an integer and clamped to the range [0,225].). Here x is a color component value (R, G, or B). The slope α controls contrast (α >1 means more contrast and 0<α<1 less contrast). For easier separation of ‘brightness’ and ‘contrast’ modifications, the formula can be written like f(x) = α(x -128) + 128 + b where b controls brightness.” Examiner notes the color component value is the element value, x is src(I), f(x) is dst(I), α is α, and β is β. Examiner further notes that this equation is rounded to an integer and clamped to the range [0,255].) .
Chadha, Huynh, and Oseiskar are considered analogous to the claimed invention because they modify images. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to generate the second space image like in Oseiskar. Thus, this would be simple substitution of one known element (generating the second space image like in Chadha in view of Huynh) for another (generating the second space image like in Oseiskar) to obtain predictable results (generate a second space image) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Chadha in view of Huynh and Oseiskar does not teach, but Cook does teach an equation to generate the second space image according to a weighted average of the RGB information. Thus, Cook teaches
wherein the generating the second space image includes generating the second space image from the first space image based on Equation[]… 4 below: [Equation 4] Y= 0.1667*R + 0.5*G + 0.3334*B (R: x’ of RGB information (x’, y’, z’) of pixel information, G: y’ of GB information (x’, y’, z’) of pixel information, B: z’ of GB information (x’, y’, z’) of pixel information, and Y: element value (x’’, y’’,z’’) after pixel information is changed) (Cook, page 1, 4th paragraph, “The luminosity method is a more sophisticated version of the average method. It also averages the values, but it forms a weighted average to account for human perception. We’re more sensitive to green than other colors, so green is weighted most heavily. The formula for luminosity is 0.21R +0.72G +0.07B”)
Chadha, Huynh, Oseiskar and Cook are considered analogous to the claimed invention because they modify images. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh and Oseiskar to generate the second space image like in Cook. Thus, this would be simple substitution of one known element (generating the second space image like in Chadha in view of Huynh and Oseiskar) for another (generating the second space image like in Cook) to obtain predictable results (generate a second space image) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Chadha, Huynh, Oseiskar, and Cook does not explicitly teach the coefficients of the equation, but Dawei teaches custom coefficients used in a weighted average. Thus Dawei teaches
[Equation 4] Y= 0.1667*R + 0.5*G + 0.3334*B (Dawei, page 12, paragraph 0068 “S101. Use a custom weighted average method to convert a three-channel color image into a single-channel grayscale image. Let the pixel values of each pixel in the color image before conversion be R(x,y), G(x,y), and B(x,y), and the grayscale value of each pixel in the converted grayscale image be Gary(x,y). The conversion rule is Gary(x,y)=a x R(x,y) + b x G(x,y) + c x B(x,y), where a+b+c=1. The values of a, b, and c are customizable.”).
Chadha, Huynh, Cook and Dawei are considered analogous to the claimed invention because they use a weighted average. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh and Cook to use the custom coefficients as in Dawei. Thus, this would be simple substitution of one known element (coefficients in Chadha, Huynh, and Cook) for another (coefficients in Dawei) to obtain predictable results (a weighted average in the form of Equation 2) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Chadha, Huynh, Oseiskar, Cook and Dawei does not explicitly teach the that the second space image is generated based on both Equations 3 and 4, but Tweakimp teaches that Equation 3 (adjusting contrast) and Equation 4 (converting the image to gray scale) can be used together. Thus Tweakimp teaches
wherein the generating the second space image includes generating the second space image from the first space image based on Equations 3 and 4 (Tweakimp, page 4-5, “For a black and white version with higher contrast in Photoshop: Select image layer of the colored version. Use CTRL+L to open the level correction. Put the middle and right arrow closer together to get a higher contrast of the colors in this range. (I set them to 0, 0.15, 200.) Add the black and white effect by putting a black and white adjustment layer on top.”).
Chadha, Huynh, Oseiskar, Cook, and Dawei are considered analogous to the claimed invention because they modify images. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh, Oseiskar, Cook and Dawei to combine Equations 3 and 4 as in Tweakimp. Thus, this would be combining prior art elements (adjusting contrast and converting to grayscale) according to known methods (Tweakimp’s method) to obtain predictable results (modify an image) (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha in view of Huynh in further view of Bell et al. (“How to add noise to image?”) (hereafter referred to as Bell).
Regarding claim 11, Chadha in view of Huynh teaches the data augmentation-based object analysis model learning apparatus of claim 10. Chadha in view of Huynh does not teach, but Bell does teach
wherein the generating the second space image includes generating the second space image by adding noise information to pixel information of the first space image based on Equation 5 below: [Equation 5] dst(I)=round(max(0,min(src(I)
±
N, 255))) (src(I): element value (x, y, z) before pixel information is changed, N: random number, dst(I) : element value (x', y', z') after pixel information is changed) (Bell, page 2, “I open a picture of a muskrat’s paw and display the image size and the value of the pixel at position (50,50). This is a RBG image ad we can be fairly sure that each colour can range from 0-255. I think you want additive Gaussian noise. To save myself the bother of writing and debugging a generator I’m using one that’s readily available, normalvariate. You will want to vary the level and spread of the noise; I’ve therefore made the mean and scale parameters. Since there are limits on the ranges of the colour values I use max and min” and Bell, page 3
PNG
media_image1.png
292
620
media_image1.png
Greyscale
Examiner notes that the max and min are bounding the pixel between 0 and 255. Examiner also notes that the add_noise function adds a random number to the pixel information.).
Chadha, Huynh, and Bell are considered analogous to the claimed invention because they add noise to an image. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to add noise to the images like in Bell. Thus, this would be simple substitution of one known element (addition of noise according to Chadha and Huynh) for another (addition of noise according to Bell) to obtain predictable results (adding noise to images) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chadha in view of Huynh in further view of Bell and Swain (“Noise in Digital Image Processing”) (hereafter referred to as Swain).
Regarding claim 13, Chadha in view of Huynh teaches the data augmentation-based object analysis model learning apparatus of claim 10. Chadha in view of Huynh does not teach, but Bell does teach
wherein the generating the second space image includes generating random number information … as much as a number of all pixels included in the first space image and generating the second space image into which noise is inserted by adding the random number information to each of the all pixels (Bell, page 2, “I open a picture of a muskrat’s paw and display the image size and the value of the pixel at position (50,50). This is a RBG image ad we can be fairly sure that each colour can range from 0-255. I think you want additive Gaussian noise. To save myself the bother of writing and debugging a generator I’m using one that’s readily available, normalvariate. You will want to vary the level and spread of the noise; I’ve therefore made the mean and scale parameters. Since there are limits on the ranges of the colour values I use max and min” and Bell, page 3
PNG
media_image1.png
292
620
media_image1.png
Greyscale
Examiner notes that the image size is 100x 117 pixels which is greater than the standard deviation. Examiner further notes that a random number is generated and added to each of the pixels)
Chadha, Huynh, and Bell are considered analogous to the claimed invention because they add noise to an image. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh to add noise to the images like in Bell. Thus, this would be simple substitution of one known element (addition of noise according to Chadha and Huynh) for another (addition of noise according to Bell) to obtain predictable results (adding noise to images) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Chadha, Huynh, and Bell do not explicitly disclose that the information is based on standard Gaussian normal distribution. Chadha, Huynh, Bell discloses generating Gaussian noise. Swain, however discloses that Gaussian noise is a standard normal distribution. Thus, Swain discloses
standard Gaussian normal distribution with an average value of 0 and a standard deviation of 100 (Swain, page 5, “Gaussian noise is a statistical noise having a probability density function equal to normal distribution, also known as Gaussian Distribution. Random Gaussian function is added to Image function to generate this noise. It is also called as electronic noise because it arises in amplifiers or detectors….The side image is a bell shaped probability distribution function which have mean 0 and standard deviation(sigma) 1.” Examiner notes that the Gaussian distribution has a standard distribution. Examiner further notes that Applicant did not use the term “standard normal …distribution” as ordinarily used (see 112(b) above). In addition, the term is not clearly defined in the specification. As such, examiner has continued to interpret “standard normal… distribution with …a standard deviation” to be its ordinary meaning that has a standard deviation of 1. ).
Chadha, Huynh, Bell, and Swain are considered analogous to the claimed invention because they add noise to an image. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Chadha in view of Huynh and Bell to use the Gaussian noise as in Swain. Thus, this would be simple substitution of one known element (addition of Gaussian noise according to Chadha, Huynh, and Bell) for another (addition of noise according to Swain) to obtain predictable results (adding Gaussian noise to images) (MPEP 2143 I. (B) Simple substitution of one known element for another to obtain predictable results).
Allowable Subject Matter
Claims 12 would be allowable over the prior art of record if the 101 and 112 rejections are overcome.
Specifically, regarding claim 12, “wherein the generating the second space image includes generating the second space image by calculating a value (Rmax-RAVG, Gmax-GAVG, Bmax-BAVG) by subtracting an element average value (RAVG, GAVG, BAVG) of each of R, G, and B of a plurality of pixels from a maximum element value (Rmax, Gmax, Bmax) among element values of each of R, G, and B of the plurality of pixels included in a size of an NxN matrix (N being a natural number equal to or greater than 3) including a first pixel at a center among pixels included in the first space image and, when any one of element values of the (Rmax-RAVG, Gmax-GAVG, Bmax-BAVG) is smaller than a preset value, performing an operation of blurring the first pixel” in conjunction with the other limitations of the claims are not taught by the prior art of record.
The closest prior art is Xin et al. (“An improved Canny edge detection algorithm for color image”) (hereafter referred to as Xin). Xin discloses a method that detects edges from color images without converting the image to gray scale first. Specifically, Xin discloses generating a second space image among element values of R, G, and B of the plurality of pixels included in a size of an NxN matrix (N being a natural number equal to or greater than 3) including a first pixel at a center among pixels included in the first space image (Xin, page 115, 2nd column). Xin also discloses a threshold used to smooth or blur the image (Xin, page 114, 1st column, last paragraph). Xin, however does not disclose calculations being performed on each of the R, G, B element values. More specifically, Xin does not disclose subtracting an average from a maximum on each R, G, and B elements of each pixel. Therefore, the prior art of record does not disclose claim 12 as a whole.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Nadernejad et al. (“Edge Detection Techniques: Evaluations and Comparisons”) also describes edge detection techniques that involve smoothing or blurring the image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN R HAEFNER whose telephone number is (571)272-1429. The examiner can normally be reached Monday - Thursday: 7:15 am - 5:15 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.H./Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights