Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The abstract of the disclosure is objected to because it does not “enable the Office and the public generally to determine quickly from a cursory inspection the nature and gist of the technical disclosure.” 37 CFR 1.72(b). Specifically, it is not clear what the technological contribution is or what the advantages are.
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
MPEP 607(III) states:
Any claim which is in dependent form but which is so worded that it, in fact, is not a proper dependent claim, as for example it does not include every limitation of the claim on which it depends, will be required to be canceled as not being a proper dependent claim; and cancellation of any further claim depending on such a dependent claim will be similarly required. The applicant may thereupon amend the claims to place them in proper dependent form, or may redraft them as independent claims, upon payment of any necessary additional fee.
Claim 20 is such a claim because it is directed to a storage medium rather than a method as in referenced claim 1, and therefore the method steps of Claim 1 need not be performed to anticipate/infringe this claim. MPEP 608.01(n)(III). While, in the interest of compact prosecution, claim 20 has been examined, claim 20 is required to be cancelled.
Claims 1, 10, and 13 are objected to for reciting “on the at.”
Claims 3 and 15 are objected to for not reciting “preset” in “matches the standard sequence.”
Claim 20 is also objected to for not reciting “non-transitory.”
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 (all claims) are rejected on the ground of nonstatutory double patenting as being unpatentable over the claims of U.S. Patent No. US11868521B2 in view of the prior art as applied below.
Both the pending claims and the conflicting patent are directed to controlling a device with gaze direction. Therefore, the conflicting patent is directed to the same problem as the present application. Further, any differences between the present claims and the claims in any of the conflicting patents are obvious in view of the prior art as applied below. It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the below prior art with any of the conflicting patents for implementation details (especially as the patent claims lack implementation details). Based on the findings herein, this is an example of “(A) Combining prior art elements according to known methods to yield predictable results.” MPEP 2143.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 10, and 13 recite “perform[ing] eye behavior recognition,” but this is unlimited functional claiming due to the wide variety of types of behaviors and ways to perform recognition. MPEP 2173.05(g). A hypothetical amendment to overcome this rejection is specifying using a specific neural network architecture or particular algorithm (i.e., a series of steps).
Claims 1, 10, and 13 recite “determin[ing] a target control instruction,” but this is also unlimited functional claiming. MPEP 2173.05(g). Specifying a type of device, a type of control, and how the determination occurs is expected overcome this rejection.
Claims 1, 10, and 13 recite “performing apparatus control,” but this is also unlimited functional claiming. MPEP 2173.05(g). Overcoming the previous issue, regarding target control, and then conforming this limitation to match that is expected overcome this rejection. In particular, the language “based on” is much broader than the specification supports; the examiner has only identified support for the apparatus control being the instruction issued by the user.
Claim 10 recites “training a recognition model by using the training sample,” but this is unlimited functional claiming. MPEP 2173.05(g).
Claim 13 recites a single instruction, but the specification does not support performing all of this with a single instruction. Reciting a plurality of instructions overcomes this issue.
Dependent claims are likewise rejected. Claim 20 is rejected as per claim 1.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 10, and 13 recite “to obtain,” but it is not clear whether obtaining is a required step. On the one hand, the plain meaning of “to obtain” is an intended use, and thus does not need to actually occur. On the other hand, the claims later recite the information as though it were obtained.
Claims 1, 10, and 13 recite “eye behavior information,” but this is new terminology. MPEP 2173.05(a). Replacing this with “gaze direction” (see claim 2) overcomes this rejection.
Claims 1, 10, and 13 recite “eye behavior information on the at least one of the facial images,” but the antecedent basis is unclear because “eye behavior information” does not use “the,” but “facial images” does.
Claims 1, 10, and 13 recite “target control instruction,” but this is new terminology. MPEP 2173.05(a).
Claims 2 and 13 recite “sequence,” but only one gaze direction is required, so it is unclear if a single instance can be a sequence.
Claims 2 and 13 recite “standard,” but it is not clear what this refers to. For example, are these standard sequences according to a standards setting body? Are they standard practice in the art?
Claims 3 and 15 recite “merg[ing] adjacent and identical gaze directions,” but adjacent and identical are distinct. Is the intent to merge with gaze directions that are adjacent, identical, or only adjacent gaze directions if they are also identical?
Claims 3 and 15 recite “sub-sequence,” but it is not clear if a single gaze direction can be a sub-sequence. It is also unclear what is meant by matching a sub-sequence to a sequence because this vitiates the pre-fix “sub.”
Claims 6 and 18 recite “eye outer contour,” but this is new terminology. MPEP 2173.05(a).
Claim 7 recites “from the eye image,” but the antecedent basis is unclear because claim 7 also recites “the eye image comprises a left eye image and a right eye image.” MPEP 2173.05(e).
Claim 9 recites “prominent,” but this is subjective. MPEP 2173.05(b)(IV).
Claim 10 is directed to “a model training method,” but recites “wherein the trained recognition model performs … .” It is unclear whether the method includes this because it is not training.
Claim 13 recites “communicating,” but this is an example of an impermissible method step in an apparatus claim. MPEP 2173.05(p)(II).
Dependent claims are likewise rejected. Claim 20 is rejected as per claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process) without significantly more.
Step 1: Claims 1 and 10 (and their dependents) recite methods, and processes satisfy Step 1 of the eligibility test.
Claim 13 (and its dependents) recite an apparatus, and machines satisfy Step 1 of the eligibility test.
Claim 20 (and its dependents) will recite a non-transitory computer-readable storage medium (note the claim objection), and manufactures satisfy Step 1 of the eligibility test.
Step 2A, prong one: All of the elements of claims 1-20 are a mental process because a person can look at a face, see where the person is looking and interpret their intent. Further, the various models are also mental processes, see example 47, claim 2, element (d) (from the July 2024 AI subject matter eligibility examples). MPEP 2106.04(a)(2)(III)(C) explains that use of a generic computer or in a computer environment is still a mental process. In particular, this section begins by citing Gottschalk v. Benson, 409 US 63 (1972). “The Supreme Court recognized this in Benson, determining that a mathematical algorithm for converting binary coded decimal to pure binary within a computer’s shift register was an abstract idea.” In Benson the Supreme Court did not separately analyze the computer hardware at issue; the specifics of what hardware was claimed is only included in an appendix to the decision.
Because there are no additional elements, no further analysis is required for Step 2A, prong two or Step 2B.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 102(a)(1) and/or (a)(2) as being anticipated by US20210405742A1 (“Xiaomi”).
1. An apparatus control method, comprising:
acquiring one or more facial images of a subject; (Xiaomi, abstract, “a head image”)
performing eye behavior recognition based on at least one of the facial images to obtain eye behavior information on the at least one of the facial images; (Xiaomi, abstract, “A method for determining a gaze position”)
determining a target control instruction based on eye behavior information on the at least one of the facial images; and (Xiaomi, [0032] “corresponding operations can be performed on the gaze position of the target user on the display screen …”)
performing apparatus control based on the target control instruction. (Xiaomi, [0032] “… for example, a control cursor on the display screen is moved to the gaze position, or a virtual button at the gaze position is selected.”)
2. The apparatus control method according to claim 1, wherein:
the eye behavior information comprises a gaze direction; and (Xiaomi, abstract, “A method for determining a gaze position”)
determining the target control instruction based on the eye behavior information on the one or more facial images comprises:
combining the gaze direction of at least one of the facial images according to an acquisition order of the one or more facial images to obtain a gaze direction sequence; (Xiaomi, [0032] “corresponding operations can be performed on the gaze position of the target user on the display screen …” Because the claim recites at least one image, the claimed sequence can be a single image, i.e., a single step sequence.)
matching the gaze direction sequence with a preset standard sequence; and (Xiaomi, [0032] “… for example, a control cursor on the display screen is moved to the gaze position, or a virtual button at the gaze position is selected.” Xiaomi’s selecting the virtual button teaches the claimed preset standard sequence. See also, [0032] “eye movements”)
determining the target control instruction according to a control instruction corresponding to the matched standard sequence. (Xiaomi, [0032] “… for example, a control cursor on the display screen is moved to the gaze position, or a virtual button at the gaze position is selected.”)
3. The apparatus control method according to claim 2, wherein matching the gaze direction sequence with the preset standard sequence comprises:
merging adjacent and identical gaze directions in the gaze direction sequence to obtain a merged sequence; and (Xiaomi, [0068] “After the gaze position is obtained each time, the operation cursor on the display screen can be moved to the gaze position.”)
determining that the gaze direction sequence matches the standard sequence in case that at least one continuous sub-sequence in the merged sequence is identical to the preset standard sequence. (Xiaomi, [0032] “… for example, a control cursor on the display screen is moved to the gaze position, or a virtual button at the gaze position is selected.” See also, [0050] “In another embodiment of the present disclosure, while determining the gaze position, another image recognition can be performed on the user image (or other types of user images obtained at the same time as the user image, for example, consecutive multiple eye images of the target user) to determine the different eye actions performed by the target user.”)
4. The apparatus control method according to claim 1, wherein performing apparatus control based on the target control instruction comprises:
controlling an application program run on a wearable apparatus based on the target control instruction, (Xiaomi, [0084] “For example, the electronic apparatus 600 may be, for example, … a mobile phone, … a messaging device, … a medical device, a fitness device, a personal digital assistant, and other terminals.”)
wherein the application program executes at least one of:
controlling operation of a smart home apparatus; (Xiaomi, [0084] “For example, the electronic apparatus 600 may be, for example, … a mobile phone, … a messaging device, … a medical device, a fitness device, a personal digital assistant, and other terminals.”)
playing or pausing display of a multimedia resource; or
controlling a camera of the wearable apparatus to capture an object of gaze. (Xiaomi, [0021] “camera on a smart hardware terminal”)
5. The apparatus control method according to claim 1, wherein performing eye behavior recognition based on at least one of the facial images to obtain eye behavior information on at least one of the facial images comprises:
extracting an eye image for at least one of the facial images; (Xiaomi, abstract “an eye image”)
extracting features from the eye image and the corresponding facial image, separately; and (Xiaomi, Fig. 1, step 103)
classifying based on the features of the eye image and the corresponding facial image to obtain the eye behavior information. (Xiaomi, Fig. 1, steps 103 and 104)
6. The apparatus control method according to claim 5, wherein extracting the eye image for at least one of the facial images comprises:
performing eye outer contour recognition on at least one of the facial images; and (Xiaomi, [0005] “The method can include determining a first space where eyes of the target user are located from a plurality of preset subspaces within the target space based on the target distance and the global image.”)
extracting, by a minimum bounding box surrounding an eye outer contour, a part located within the minimum bounding box as the extracted eye image. (Xiaomi, [0005] “The method can include determining a first space where eyes of the target user are located from a plurality of preset subspaces within the target space based on the target distance and the global image.” Fig. 4B shows that the preset spaces teach the claimed bounding boxes.)
7. The apparatus control method according to claim 5, wherein:
the eye image comprises a left eye image and a right eye image, and extracting features from the eye image and the corresponding facial image separately comprises: (Xiaomi, [0026] “The eye image may be a high resolution image including the eyes of the target user.” Note the plural “eyes”)
inputting the left eye image and the right eye image into corresponding first feature extraction networks for feature extraction, (Xiaomi, [0041] “The convolutional layer is configured to extract image features from the user image, … the eye image, different convolution kernels can be used to extract partial features.”)
wherein each of the first feature extraction networks extracts at least one of a pupil position feature or an eye shape feature; and (Xiaomi, [0041] “In the embodiments of the present disclosure, for the global image, the head image, and the eye image, different convolution kernels can be used to extract partial features.”)
inputting the facial image into a second feature extraction network for feature extraction, to obtain a feature of the facial image, wherein a convolution kernel of the second feature extraction network is larger than a convolution kernel of the first feature extraction network. (Xiaomi, [0041] “For example, for the eye image with the aspect ratio of 16:9 and 48 million pixels, a larger convolution kernel, for example, a convolution kernel containing a pixel matrix of 100*100, can be used for feature extraction.”)
8. The apparatus control method according to claim 7, wherein classifying based on the features of the eye image and the corresponding facial image to obtain the eye behavior information comprises:
inputting features extracted from the left eye image and the right eye image into a first fully connected layer for vector space mapping to obtain an output of the first fully connected layer; (Xiaomi, [0065] “the fully connected layer so that a target dimension is the number of areas divided by the first hierarchy” Xiaomi teaches separate fully connected layers for each of the hierarchies)
inputting the feature of the facial image into a second fully connected layer for vector space mapping to obtain an output of the second fully connected layer; (Xiaomi, [0065] “the fully connected layer so that the target dimension is the number of areas divided by the second hierarchy”)
inputting the output of the first fully connected layer and the output of the second fully connected layer into a third fully connected layer for vector space mapping to obtain an output of the third fully connected layer; and (Xiaomi, [0066] “Then, the user image of user A and the screen coordinates corresponding to the gaze position of user A in the third hierarchy of the display screen are input into the model for a third training”)
activating the output of the third fully connected layer by using an activation function layer to classify and obtain the eye behavior information. (Xiaomi, [0067] “This process can be understood as obtaining an approximate range of the gaze position of the target user after the first input process, and then narrowing down the range layer by layer in the subsequent input processes, and finally obtaining an accurate range.” Xiaomi’s accurate range teaches the claimed activation layer.)
9. The apparatus control method according to claim 7, wherein each of the first feature extraction network and the second feature extraction network comprises one or more convolution blocks connected in sequence, and
each of the convolution blocks comprises:
a convolution layer that performs a convolution operation by using a convolution kernel; and (Xiaomi, [0048] “the number of convolution kernels in the convolution layer can be determined according to the actual training situation”)
a pooling layer that takes a most prominent feature output by the convolution layer in the convolution block as an input of a next convolution block. (Xiaomi, [0048] “the dimension reduction operation of the pooling layer, further convolution operations”)
Claims 10-12 are rejected as per their counterpart claims. Xiaomi also teaches the claimed training, see, e.g., the training processes that begin at [0040] and [0065].
Claims 13-19 are rejected as per their counterpart claims. Xiaomi also teaches the claimed electronic apparatus, see, e.g., claim 18.
Claim 20 is rejected as per claim 1. Xiaomi also teaches the claimed storage medium, see, e.g., claim 17.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US11132543B2 – titled “Unconstrained appearance-based gaze estimation”
US11636609B2 – titled “Gaze determination machine learning system having adaptive weighting of inputs”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID ORANGE whose telephone number is (571)270-1799. The examiner can normally be reached Mon-Fri, 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ORANGE/ Primary Examiner, Art Unit 2663