DETAILED ACTION
Contents
Notice of Pre-AIA or AIA Status 2
Response to Amendment 2
Response to Arguments 2
Claim Rejections - 35 USC § 101 5
Claim Rejections - 35 USC § 102 6
Claim Rejections - 35 USC § 103 11
Conclusion 14
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is responsive to applicant’s amendment and remarks received on 3/30/26. Claims 1-20 are currently pending. Claims 12-20 are withdrawn from consideration.
Response to Arguments
Applicant's arguments filed on 3/30/26 have been fully considered but they are not persuasive. Applicant argues that Liu does not disclose a “first image comprising a high confidence label” (see pg. 8). This argument is not persuasive because Liu second or known images can have an associated certainty probability above a threshold which corresponds to the high confidence label as seen within col. 3, lines 1-50, col. 6, lines 40-60, col. 7, lines 40-61.
Applicant further asserts Liu does not teach an image comprising a low confidence label (see pg. 9). This argument is not considered persuasive because Liu teaches an image may be classified by a learning model with a certainty probability below a threshold as seen within col. 3, lines 1-35, col. 6, lines 40-60.
Applicant further contends that Liu does not teach selecting a low confidence labeled second image that matches the high confidence image (see 9). This argument is not considered persuasive because within Liu it is taught that the user is instructed to select images that look like the known reference image such as an unknown image. The user evaluates and selects if the unknown or low confidence image matches a known or high confidence reference image as seen within col. 7, lines 50-66, col. 8, lines 1-30, col. 8, lines 1-30.
Applicant argues Liu does not disclose modifying the low confidence label of the second image (see pg. 9). This argument is not considered persuasive because Liu discloses user selection to infer unknown characteristic, then annotates the image to indicate the classification. The changing of an image from unknown to annotated is a modification of the low confidence label image. This is shown within col. 7, lines 1-60, col. 9, lines 1-30.
Applicant contends Liu does not disclose high/low confidence framework as described within the claim 1 (see pg. 9). This argument is not considered persuasive since the limitation as seen above in the previous responses also even if the terminology differs, the functional arrangement is disclosed.
Regarding claims 2-3, 9, the applicant argues that the limitation within claim 1 are not disclosed, thus claims 2-3, 9 are not disclosed as well. This argument is not persuasive since claim 1 is disclosed by the prior art listed and the arguments by the applicant are not convincing.
Regarding claim 4, the applicant asserts that the Mahmoud is non-analogous. This argument is not persuasive because Mahmoud is utilized to teach the limitation cited and does not need to teach the same framework as Liu. “[O]bviousness must be determined in light of all the facts, and there is no rule that a single reference that teaches away will mandate a finding of nonobviousness. Likewise, a given course of action often has simultaneous advantages and disadvantages, and this does not necessarily obviate motivation to combine. Medichem, S.A. v. Rolabo, S.L., 437 F.3d 1157, 1165 (Fed. Cir. 2006). Applicant further cites that there is no rationale and would change the principle of operation. This is not considered persuasive since Mahmoud is merely supplying a known class of visual image data and the substitution would have been predictable use of prior-art image classification content in Liu’s captcha framework. See In re Ratti, 270 F.2d 810 (CCPA 1959) (If the proposed modification or combination of the prior art would change the principle of operation of the prior art invention being modified, then the teachings of the references are not sufficient to render the claims prima facie obvious.); see also M.P.E.P. § 2143.01(VI).
Regarding claim 5, Applicant asserts the Welinder confidence labels are annotator confidence labels not claimed image label states. This argument is not persuasive because the claim 5 does not high-confidence, low-confidence or no-label states to be generated by any source. The combination of Liu and Welinder teaches all of the limitations as cited.
Regarding claim 6, applicant argues Liu only uses user selections, not a maintained list of labels entered by users and does not identify matching user-entered label for a low confidence image. This argument tis not persuasive because Liu discloses all elements as seen within the rejection of claim 6.
Regarding claim 7-8, applicant argues that Liu does not disclose the “all match” or “none match”. This argument tis not persuasive because Liu discloses all elements as seen within the rejection of claims 7-8.
Regarding claim 10, applicant argues Sun’s confidence score is not a high-confidence label attached to images. This argument is not persuasive because the limitation is taught within Sun. Applicant further argues that there is no rationale to select 90-100%. This argument is not considered persuasive since it is ordinary to one ordinary skilled in the art to improve reliability of the system and for the reasons as stated within the motivational statement to combine for claim 10.
Claim Rejections - 35 USC § 101
Applicant argues that the claims are not a mental process (see pgs. 6-8). Applicant’s argument is not persuasive because claim 1 does not recite any specific improvement to image storage, display hardware, etc. The claim broadly cites presenting images, receiving a user selection and changing a label based on the selection. The limitations amount to an abstract idea of comparing visual information and updating classification confidence. The recitation of a computer display device is a generic computer implementation of an abstract idea and does not remove the claim from the mental processing.
Applicant further argues that the claims are integrated into a practical application (see pg. 6-8). This argument is not considered persuasive since claim 1 merely uses a generic computer display device to present images and receive a user’s selection. Applicant further argues that the claims recite an inventive concept (see pg. 6-8). This argument is not considered persuasive since the additional computer elements merely perform conventional data presentation, collection and updating functions.
Applicant further argues that the dependent claims add technical features (see pg. 6-8). This argument is not considered persuasive since the limitations recite language that does not claim a specific improvement to the machine learning model or computer functionality.
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows. Claim 1 recites an abstract idea that can be performed mentally, with no integration into a practical application, and with no inventive concept. Claims 2-11 also state abstract ideas with no practical application or significantly more. Thus all claims are rejected as non statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless - (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 6-9, 11 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Liu et al (US 9,760,700 B2). Regarding claim 1, Liu discloses A process comprising:
displaying a first image on a computer display device, the first image comprising a high confidence label (see col. 7, lines 50-67; FIG. 2 depicts an example image based CAPTCHA challenge 110 according to example embodiments of the present disclosure. As shown, CAPTCHA challenge 110 includes instructions 112 and an associated reference image 114 that is known to depict a dog. Instructions 112 prompt the user to select all the images that “look like” reference image 114. CAPTCHA challenge 110 further includes known images 116 and negative known images 118. Known images 116 are images that are known to depict a dog, and negative known images 118 are images that are known to not depict a dog. Although each negative known image 118 depicts cats, it will be appreciated that negative known images may depict one or more other objects, items, plants, animals, humans, etc. CAPTCHA challenge 110 further depicts an unknown image 120. Unknown image 120 can be an image wherein the contents of the image are unknown);
displaying a plurality of additional images on the computer display device, one or more of the plurality of additional images comprising a low confidence label (see col. 7, lines 50-67, col. 3, lines 5-45; FIG. 2 depicts an example image based CAPTCHA challenge 110 according to example embodiments of the present disclosure. As shown, CAPTCHA challenge 110 includes instructions 112 and an associated reference image 114 that is known to depict a dog. Instructions 112 prompt the user to select all the images that “look like” reference image 114. CAPTCHA challenge 110 further includes known images 116 and negative known images 118. Known images 116 are images that are known to depict a dog, and negative known images 118 are images that are known to not depict a dog. Although each negative known image 118 depicts cats, it will be appreciated that negative known images may depict one or more other objects, items, plants, animals, humans, etc. CAPTCHA challenge 110 further depicts an unknown image 120. Unknown image 120 can be an image wherein the contents of the image are unknown…. the models can be further configured to determine a certainty probability corresponding to a degree of certainty that the output of the model is correct. In some implementations, a characteristic of an image can be unknown if the image has an associated certainty probability below a threshold. For instance, the first image having one or more unknown characteristics can be an image having a certainty probability below a threshold);
receiving input from a user, the input comprising a selection of a second image from the additional images comprising the low confidence label that matches the image comprising a high confidence label (see col. 7, lines 50-col. 8, lines 18; FIG. 2 depicts an example image based CAPTCHA challenge 110 according to example embodiments of the present disclosure. As shown, CAPTCHA challenge 110 includes instructions 112 and an associated reference image 114 that is known to depict a dog. Instructions 112 prompt the user to select all the images that “look like” reference image 114. CAPTCHA challenge 110 further includes known images 116 and negative known images 118. Known images 116 are images that are known to depict a dog, and negative known images 118 are images that are known to not depict a dog. Although each negative known image 118 depicts cats, it will be appreciated that negative known images may depict one or more other objects, items, plants, animals, humans, etc. CAPTCHA challenge 110 further depicts an unknown image 120. Unknown image 120 can be an image wherein the contents of the image are unknown. As shown in FIG. 2, unknown image 120 depicts a lizard. (28) As described above, upon receiving CAPTCHA challenge 110, a user can follow the instructions and select one or more images that, in the user's opinion, “look like” reference image 114. The user's selections can determine whether the user is granted access to the resources or service provided by a resource provider. For instance, if the user “passes” the challenge, the user can be granted access to the resource or service. For instance, the user may pass the challenge if the user selects the correct combination of images, or close to the correct combination of images. In CAPTCHA challenge 110, the user will pass if the user selects each known image 116 and no negative known images 118. In some implementations, if the user does not “pass,” the user can be presented a different CAPTCHA challenge.); and
modifying the low confidence label of the second image (see col. 4, lines 45-col. 5, lines 30; In this manner, if enough users (e.g. a number of users above a threshold) select the first image in response to receiving the verification challenge, it can be inferred that the image depicts a tiger. If the number of users who select the first image is below the threshold, it can be inferred that the first image does not depict a tiger. In some implementations, the inference can be made based at least in part on a percentage of users who select the image. As another example, in implementations wherein a verification challenge includes instructions prompting a user to locate and interact with a portion of the first image that depicts a particular object, the user selections can be used to determine the placement of the object in the first image. (15) In some implementations, the user responses to the verification challenge can be used to confirm the known images. For instance, the user responses to the verification challenge may be used to verify that the known characteristic of the known images are correct. In this manner, if enough user responses associated with a known image indicate that the known characteristics are incorrect, the image can be removed from the set of known images. As another example, the known characteristic may be adjusted or refined based at least in part on the user responses.).
Regarding claims 2, 6-9, 11, Liu discloses permitting the user to access a processor-based system when the user selects the second image that matches the image comprising the high confidence label (see col. 3, lines 50-col. 4, lines 35);
maintaining a list of labels of the second image that were entered by a plurality of users (see col. 4, lines 35-67);
identifying labels of the second image that were entered by the plurality of users that match (see col. 4, lines 35-67); and
modifying the low confidence label of the second image when a number or percentage of the labels of the second image that were entered by the plurality of users and that match crosses a threshold (see col. 4, lines 35-67);
plurality of additional images match the first image; and permitting the user to access a processor-based system only when the user indicates that all the additional images match the first image (see col. 4, lines 35-67);
none of the plurality of additional images match the first image; and permitting the user to access a processor-based system only when the user indicates that none of the plurality of additional images match the first image (see col. 4, lines 35-67);
using the plurality of additional images to train a machine learning algorithm (see col. 4, lines 35-67);
modifying the low confidence label of the second image comprises increasing the low confidence label of the second image (see col. 13, lines 1-50).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimedinvention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (US 9,760,700 B2) in view of Vondrick et al (UC: “Video Annotation and Tracking with Active Learning”).
Regarding claim 3, Liu teaches all elements as mentioned above in claim 1. Liu does not teach expressly plurality of images comprises video data.
Vondrick, in the same field of endeavor, teaches plurality of images comprises video data (see abstract, section 1, 2; video annotation, key frames).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Liu to utilize the cited limitations as suggested by Vondrick. The suggestion/motivation for doing so would have been to enhance the system through excellent performance at a small fraction of the cost (see abstract). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Liu, while the teaching of Vondrick continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (US 9,760,700 B2) in view of Mahmoud et al (US 10,262,198 B2).
Regarding claim 4, Liu teaches all elements as mentioned above in claim 1. Liu does not teach expressly plurality of images comprises signs of a sign language.
Mahmoud, in the same field of endeavor, teaches plurality of images comprises signs of a sign language (see abstract).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Liu to utilize the cited limitations as suggested by Mahmoud. The suggestion/motivation for doing so would have been to enhance the system by using robust local features to recognize poses for unseen signs (see col. 7, lines 35-67). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Liu, while the teaching of Mahmoud continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (US 9,760,700 B2) in view of Welinder et al (US 9,704,106 B2).
Regarding claim 5, Liu teaches all elements as mentioned above in claim 1. Liu does not teach expressly plurality of images is stored in a database, the database comprising images with high confidence labels, images with low confidence labels and images with no labels.
Welinder, in the same field of endeavor, teaches plurality of images is stored in a database, the database comprising images with high confidence labels, images with low confidence labels and images with no labels (see abstract).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Liu to utilize the cited limitations as suggested by Welinder. The suggestion/motivation for doing so would have been to enhance determine the accuracy of the data annotation (see col. 6, lines 25-40). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Liu, while the teaching of Welinder continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (US 9,760,700 B2) in view of Sun et al (US 9,858,496 B2).
Regarding claim 10, Liu teaches all elements as mentioned above in claim 1. Liu does not teach expressly a certainty in a range of approximately 90% to 100%.
Sun, in the same field of endeavor, teaches a certainty in a range of approximately 90% to 100% (see col. 8, lines 15-40).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Liu to utilize the cited limitations as suggested by Sun. The suggestion/motivation for doing so would have been to increase the accuracy of the object detection (see col. 1, lines 1-30). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Liu, while the teaching of Sun continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD PARK. The examiner’s contact information is as follows:
Telephone: (571)270-1576 | Fax: 571.270.2576 | Edward.Park@uspto.gov
For email communications, please notate MPEP 502.03, which outlines procedures pertaining to communications via the internet and authorization. A sample authorization form is cited within MPEP 502.03, section II.
The examiner can normally be reached on M-F 9-6 CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer, can be reached on (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD PARK/
Primary Examiner, Art Unit 2666
Prosecution Insights