DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9, and 12-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20220005303) in view of Zhang (English translation of CN115311700), and Diaz ("Shufflefacenet: A lightweight face architecture for efficient and highly-accurate face recognition." In Proceedings of the IEEE/CVF international conference on computer vision workshops, 2019.).
Regarding claim 1:
Kim discloses: a robot service system (Kim discloses a building management robot and a building management system in which a robot communicates with one or more servers and provides building related services to persons recognized by the robot. See FIGS. 4, and 7- 9) comprising:
a server configured to store a plurality of face recognition models (Kim discloses an AI server having a memory that includes model storage unit. The model storage unit stores a learned model or ANN, and the robot may receive data corresponding to a learned recognition model from the server, store the received data, and use it to recognize a person. See FIGS. 2 and 4 and ¶ [0078] – [0083]);
Kim does not expressly teach that the server stores a plurality of plurality of face recognition models having different degrees of light-weighting.
However, in the same field of endeavor, Zhang teaches: a server stores a plurality of plurality of face recognition models configured for different computing capabilities and data reading capabilities (Zhang teaches a configuration server that saves and manages candidate facial recognition algorithm models. The configuration server maintains candidate facial recognition models 1-3, which are trained for different hardware configurations and model requirements (Zhang ¶¶ [69] – [72], [76] – [82], [95] – [98] and [129] – [140]));
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Zhang by including: a server stores a plurality of plurality of face recognition models in order to adapt to different hardware configurations of a particular facial recognition device.
Kim in view of Zhang does not specifically teach that the plurality of face recognition models having different degrees of light-weighting.
However, in the same field of endeavor, Diaz teaches: plurality of face recognition models having different degrees of light-weighting (Diaz teaches that face network requires 138 million parameters, which some platforms such as robots are unable to deploy a real-time application that is capable of these computations. Therefore, Diaz proposes a new lightweight architecture named ShuffleFaceNet, that extends the extremely efficient network ShuffleNetV2 [19] to the domain of face recognition (Diaz “1. Introduction”). Diaz further teaches “We design an efficient and accurate light weight face architecture, with four different complexity levels. The resulting ShuffleFaceNet models are less than 20 MB of size and have an actual inference CPU time of about 37 ms, which is suitable for deploying on real-time ap plications, as well as, mobile and embedded devices. (Diaz “1. Introduction”, page 2722, left col.); Also see tables 1-4 with different levels of complexity (different degrees of light-weighting);
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Kim and Zhang to incorporate the teachings of Diaz by including: plurality of face recognition models having different degrees of light-weighting in order to adapt to different hardware configurations of a particular facial recognition device, which only experiences a marginal decrease in accuracy but offer significant savings in computational cost.
Kim further discloses: a robot execute a face recognition function using a face recognition model (Kim teaches that its robot recognizes a person from image data using a known face recognition algorithm. Kim further states that the robot may recognize the person using a recognition model stored in robot memory and that the model may be an artificial neural network trained by the robot or server (Kim FIGS. 5-6 and ¶¶ [0174] – [00177])) configured to:
Zhang further teaches: the face recognition model selected, based on hardware environment information of the robot, from the plurality of face recognition models (Zhang teaches obtaining hardware configuration information from facial recognition endpoint (access control device) including processor type, memory capacity, camera parameters, and imaging component parameters (Zhang ¶¶ [64] - [67]); Zhang also lists Models 1-3 with different hardware configuration and selects model 2 because its associated hardware configuration most closely matches the endpoint (Zhang ¶¶ [81] – [82]). Zhang then sends the selected model to the endpoint, which downloads and installs the model (Zhang ¶ [82]));
Kim further teaches: identify, based on the face recognition function, a user (Kim teaches that the robot executes facial recognition using a recognition model stored in robot memory and recognizing the person (Kim ¶¶ [0147], [0173] – [0177]. Zhang teaches transmitting the selected model to the endpoint for downloading, installing, and executing (Zhang ¶ [82])); and
provide, based on identifying the user, a service (Kim teaches providing a service based pm the authentication level of a person (Kim ¶¶ [0208] – [0217] and FIG. 9)).
Regarding claim 2:
Zhang further teaches: the robot is further configured to: transmit, to the server, the hardware environment information of the robot; receive the selected face recognition model from the server; and install, on the robot, the selected face recognition model; and the server is further configured to: select, based on the hardware environment information, the selected face recognition model from the plurality of face recognition models; and transmit, to the robot, the selected face recognition mode (Zhang ¶¶ [69] – [72], [76] – [82], [95] – [98] and [129] – [140]).
Regarding claim 3:
Zhang further teaches: he hardware environment information comprises hardware specification information of the robot (Zhang ¶¶ [65] – [66], [73], [81], and [131] ).
Regarding claim 4:
Zhang further teaches: the hardware environment information comprises at least one of: a type of application installed on the robot; a remaining capacity of a memory or a storage of the robot; or performance information indicative of actual driving performance of the robot. (Zhang ¶¶ [65] – [66], [73], [81], and [131] ).
Regarding claim 5:
Zhang further teaches: the hardware environment information comprises target performance information associated with the face recognition function (Zhang ¶¶ [87] –[88] and [159] – [161] ).
Regarding claim 6:
Zhang further teaches: the server is further configured to: store a lookup table comprising the plurality of face recognition models and corresponding feature information, wherein the feature information comprises, for each face recognition model of the plurality of face recognition models, at least one of a size, a throughput, memory usage, or performance; and select the selected face recognition model further based on the feature information. (Zhang ¶¶ [87] –[88] and [159] – [161]. Diaz further teaches Diaz “We design an efficient and accurate light weight face architecture, with four different complexity levels. The resulting ShuffleFaceNet models are less than 20 MB of size and have an actual inference CPU time of about 37 ms, which is suitable for deploying on real-time ap plications, as well as, mobile and embedded devices. (Diaz “1. Introduction”, page 2722, left col.); Also see tables 1-4 with different levels of complexity (different degrees of light-weighting).
Regarding claim 7:
Diaz further teaches: the hardware environment information comprises target performance information associated with the face recognition function (Diaz teaches that some approaches for lightweight models include compressing or accelerating pre-trained DNN or CNN networks by using techniques such as pruning [9], knowledge distillation [11], low-rank approximation [41] and quantization [16]. (Diaz “1. Introduction”)).
Regarding claim 8:
Kim and Zhang further teach: the robot is further configured to, based on executing the face recognition function: detect, based on an image of a face recognition target, facial feature information; and verify, based on the facial feature information, an identity of the face recognition target (Kim FIG. 9 and Zhang ¶ [166]).
Regarding claim 9:
Kim and Zhang further teach: the robot is further configured to: detect, based on an image of a face recognition target, facial feature information; transmit, to the server, the facial feature information; and the server is further configured to: store user information comprising facial feature information for a plurality of pre-registered users (Kim discloses a service robot that obtains image data of a person and detects facial feature information for recognizing the user. Kim further discloses communication between the robot and server, and user information stored in a database for determining the recognized user’s authentication level. Specifically Kim teaches server 200a may add a first record RECORD1 including the first identification information D_INFO1 and the first image feature IMAGE1 to the database, based on the received first identification information D_INFO1 and the image feature data IMAGE1, IMAGE2 and IMAGE3 (Kim FIG. 6-9 and ¶¶ [0192] – [0205], D_INFO1 )); and verify, based on comparing the facial feature information received from the robot to the facial feature information of the plurality of pre-registered users, an identity of the face recognition target (Kim ¶ [0120]. Zhang also teaches comparing the image features of the facial image to be recognized with the image features of the first facial image. If the similarity between the two image features is greater than the preset threshold, it is determined whether the person A is a registered person and can enter the industrial park smoothly. (Zhang ¶ [138]).
Regarding claim 12:
Kim further teaches: wherein the robot is configured to, based on the identified user being an unregistered user, provide the service, wherein the service comprises at least one of: a greeting service configured to guide the unregistered user, or a surveillance service configured to surveil the unregistered user (Kim ¶ [0012], ¶ [0231] – [0233], and ¶ [0280]).
Regarding claim 13:
Kim further teaches: wherein the robot is configured to, based on the identified user being an unregistered user, provide the service, wherein the service comprises at least one of: a greeting service configured (Kim ¶ [0217]).
Regarding claims 14-20:
Claim 14 is rejected in the same manner as claim 1.
Claim 15 is rejected in the same manner as claim 2.
Claim 16 is rejected in the same manner as claim 3-5.
Claim 17 is rejected in the same manner as claim 7.
Claim 18 is rejected in the same manner as claim 8-9.
Claim 19-20 is rejected in the same manner as claim 12-13.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20220005303) in view of Zhang (English translation of CN115311700), and Diaz ("Shufflefacenet: A lightweight face architecture for efficient and highly-accurate face recognition." In Proceedings of the IEEE/CVF international conference on computer vision workshops, 2019.) and Mahalingam ("Can discriminative cues aid face recognition across age?." In 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), pp. 206-212. IEEE, 2011.).
Regarding claim 10:
Kim further discloses detect, based on the image, additional attribute information comprising at least one of age or gender; and transmit, to the server, the additional attribute information (Kim ¶ [0018], ¶ [0268], ¶ [0269])).
However, Kim in view of Zhang and Diaz does not specifically teach: the server is further configured to: select, based on the additional attribute information, a candidate group from the plurality of pre-registered users; and verify the identity of the face recognition target by comparing the facial feature information received from the robot to a portion of the facial feature information corresponding to the candidate group.
Nonetheless, in the same field of endeavor, Mahalingam teaches: the server is further configured to: select, based on the additional attribute information, a candidate group from the plurality of pre-registered users; and verify the identity of the face recognition target by comparing the facial feature information received from the robot to a portion of the facial feature information corresponding to the candidate group (Mahalingam teaches using discriminative cues including gender and age of a test face image, to prune the face recognition search space before recognition (abstract). Mahalingam further teaches extracting gender and age group information from the face image and using that information to reduce the candidate search space for efficient face recognition (FIG. 1 and C. Contribution)).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Kim in view of Zhang and Diaz to incorporate the teachings of Deyle by including: select, based on the additional attribute information, a candidate group from the plurality of pre-registered users; and verify the identity of the face recognition target by comparing the facial feature information received from the robot to a portion of the facial feature information corresponding to the candidate group in order to reduce the computation time by reducing the search space.
Claim(s) 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20220005303) in view of Zhang (English translation of CN115311700), and Diaz ("Shufflefacenet: A lightweight face architecture for efficient and highly-accurate face recognition." In Proceedings of the IEEE/CVF international conference on computer vision workshops, 2019.) and Deyle (US 20170225334).
Regarding claim 11:
Kim further discloses the server is further configured to: collect location information of the robot (Kim teaches the robot recognizing the user by face and providing services and detecting the location of the robot (Kim ¶ [0139])).
However, Kim in view of Zhang and Diaz does not specifically teach: manage, based on the location information, behavioral information of a user recognized via the face recognition function.
Nonetheless, in the same field of endeavor, Deyle teaches: manage, based on the location information, behavioral information of a user recognized via the face recognition function (Deyle teaches a central system managing mobile security robots, collecting or using robot/location information, and managing information identifying an individual’s historical movements or behaviors (Deyle ¶ [0007], ¶ [0115], ¶ [0120], ¶ [0137], ¶ [0253], and ¶ [0270]).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Kim in view of Zhang and Diaz to incorporate the teachings of Deyle by including: manage, based on the location information, behavioral information of a user recognized via the face recognition function in order to manage recognized users for security, access, or service purposes.
Prior Art not relied on
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lee (US 20230394875) teaches heavy facial recognition model and the lightweight recognition model exhibit a difference in complexity of about 20 times.
Chen (US 20230267709) teaches heavyweight models tend to perform better than lightweight models. The terms “heavyweight” and “lightweight” refer to the amount of computational resources needed by a corresponding model (and thus may dictate where the corresponding model resides). Some lightweight models enable facial recognition to be performed on computing devices with limited computational resources, such as mobile phones, tablet computers, and wearable electronic devices. These lightweight models may have carefully designed architectures that experience only a marginal decrease in accuracy but offer significant savings in computational cost.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASSIM MAHROUKA whose telephone number is (571)272-2945. The examiner can normally be reached Monday-Thursday 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASSIM MAHROUKA/Primary Examiner, Art Unit 2665
Prosecution Insights