Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,254,687 to Mishra (present inventor) et al.
The conflicting claims are:
Claims 1-20 in present application 19/056,167
Claims 1-20 in US Patent No. 12,254,687.
Table 1 provided below is the comparative mapping of the limitations of claims 1-20 of the present application when compared against the limitations of claims 1-20 of Patent No. 12,254,687.
Present Application 19/056,167
US Patent No. 12,254,687
1. A method of determining an operating status of one or more hardware components of a facial recognition system of a mobile device, the method comprising: engaging a dot projector and an infrared ("IR") camera of the mobile device; capturing an image of the dot projector and the IR camera; comparing the captured image to a predetermined test image; and based on the results of the comparison, determining the operating status of at least one of the dot projector and the IR camera.
1. A method of determining an operating status of one or more hardware components of a facial recognition system of a mobile device, the method comprising: engaging the facial recognition system of the mobile device to put the mobile device into a test mode, wherein the facial recognition system comprises a dot projector and an infrared (“IR”) camera; while the mobile device is in the test mode, capturing an image of the dot projector and the IR camera; comparing the captured image to a predetermined test image; and based on the results of the comparison, determining the operating status of at least one of the dot projector and the IR camera.
2. The method of claim 1, wherein engaging the infrared ("IR") camera includes turning on a front camera of the mobile device and putting the front camera into portrait mode.
2. The method of claim 1, wherein engaging the facial recognition system of the mobile device includes turning on a front camera of the mobile device and putting the front camera into portrait mode.
3. The method of claim 1, wherein the captured image and the predetermined test image are digital images.
3. The method of claim 1, wherein the captured image and the predetermined test image are digital images.
4. The method of claim 1, wherein the step of comparing the captured image to the predetermined image includes comparing a first portion of the captured image with a corresponding first portion of the predetermined test image.
4. The method of claim 1, wherein the step of comparing the captured image to the predetermined image includes comparing a first portion of the captured image with a corresponding first portion of the predetermined test image.
5. The method of claim 4, wherein the first portion of the captured image receives IR illumination from the dot projector of the mobile device.
5. The method of claim 4, wherein the first portion of the captured image receives IR illumination from the dot projector of the mobile device.
6. The method of claim 5, wherein the step of comparing the captured image to the predetermined image includes comparing a second portion of the captured image with a corresponding second portion of the predetermined test image.
6. The method of claim 5, wherein the step of comparing the captured image to the predetermined image includes comparing a second portion of the captured image with a corresponding second portion of the predetermined test image.
7. The method of claim 6, wherein the second portion of the captured image receives IR illumination from the IR camera of the mobile device.
7. The method of claim 6, wherein the second portion of the captured image receives IR illumination from the IR camera of the mobile device.
8. The method of claim 1, wherein the step of comparing the captured image to the predetermined image includes comparing a luminance value of a pixel or group of pixels of the captured image with a luminance value of a corresponding pixel or group of pixels of the predetermined test image.
8. The method of claim 1, wherein the step of comparing the captured image to the predetermined image includes comparing a luminance value of a pixel or group of pixels of the captured image with a luminance value of a corresponding pixel or group of pixels of the predetermined test image.
Claims 9, 17 and their dependents correspond to claim 1 and its dependents and are similarly rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-7, 9, 11-15, 17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (US Patent Pub. No. 2021/0176157 A1, published 2021) in view of Radiant (NPL, “Measuring Near-Infrared (NIR) Light Sources for Effective 3D Facial Recognition”, published 2019, pdf attached).
Regarding claim 1, Huang teaches a method of determining an operating status of one or more hardware components of a display of a mobile device (Para. 30, “As illustrated in FIG. 1, the transmission structure 120 is configured to carry the automatic test devices 110 and the corresponding devices under test DUT.”), the method comprising: engaging a display of the mobile device (Para. 100, “In the block 620, the automatic test device orders the device under test play the on-board sample video that is produced in the block 615. At the same time, the camera captures the content that is displayed by the device under test and corresponds to the on-board sample video, and generates test photos.”); capturing an image of the display (Para. 93, “As a result, the preliminary sample photos are display illustrations that captured the illustrations displayed by the device under test at some timing by the camera”); comparing the captured image to a predetermined test image (Para. 101, “In the block 625, the computing equipment compares the test photos in the block 620 with the preliminary sample photos in the block 615, to determine whether the test photos match the preliminary sample photos.”); and based on the results of the comparison, determining the operating status of the display (Para. 113, “In some embodiments, in the block 625, since the test photos are color photos such as that shown in FIG. 5B, whether the colors are abnormal is determined by comparing the color sections e of the test photos with the preliminary sample photos correspondingly.”).
Huang does not explicitly disclose determining an operating status of a facial recognition system of a mobile device by engaging a dot projector and an infrared camera and capturing an image of it. However, their method is towards the automatic testing of components of electronic devices, including mobile devices.
Radiant teaches determining an operating status of one or more hardware components of a facial recognition system of a mobile device (Pg. 3, “This paper discusses how NIR light can be used in 3D facial recognition systems, and the methods for measuring and testing NIR emitters to help ensure they are accurate and effective for use in consumer electronics applications.”), the method comprising: engaging a dot projector and an infrared ("IR") camera of the mobile device (Pg. 5, “The use of image-based NIR measurement systems (for example, a sensor-based radiometric camera) for NIR source measurement can limit this complexity by capturing and measuring all emission points produced by a DOE across a large spatial area.”, in order to image and capture all emission points produced by a DOE it must be engaged); capturing an image of the dot projector (Pg. 5 above); and determining the operating status of at least one of the dot projector and the IR camera (Pg. 6, “Advanced NIR measurement systems use Fourier optics to capture a full cone of data in a single image, to measure radiant intensity of an entire NIR light source. Identifying irregularities, peak emission, hot spots, and other issues over angular space”).
Radiant does not explicitly disclose capturing an image of the dot projector and the IR camera. However, they do disclose testing both of these components as a paired unit (Fig. 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Huang to incorporate the teachings of Radiant to include determining an operating status of a facial recognition system of a mobile device by engaging a dot projector and an infrared camera and capturing an image of it. Huang discloses an automatic device testing system which uses a camera to image the device when it is engaged in a test mode and compares the test image to predetermined images to determine an operating status. However, Huang does not disclose performing this analysis on a facial recognition system. Radiant teaches a method for verifying a facial recognition systems operating status by capturing images of the components when they are engaged to evaluate the measured data for irregularities. They also disclose determining the intensity of the IR emissions against defined tolerances. One of ordinary skill in the art would have understood that applying Radiant’s known facial recognition emitters and testing to Huang’s method of using a test image of the device compared to a stored template of the device is a combination of known elements yielding predictable results. This would reliably achieve both Huang’s stated goal of replacing costly manual testing with automated testing (Para. 4) by extending the capabilities of the system, and Radiant’s stated need for effective methods to measure the accuracy of NIR emitters to avoid performance issues (Pg. 4).
Regarding claim 3, Huang as modified above teaches all of the elements of claim 1, as stated above, as well as wherein the captured image and the predetermined test image are digital images (Para. 84, “At the same time, images that displayed by the device under test are captured by the camera and featured photos are generated.”).
Regarding claim 4, Huang as modified above teaches all of the elements of claim 1, as stated above, as well as wherein the step of comparing the captured image to the predetermined image includes comparing a first portion of the captured image with a corresponding first portion of the predetermined test image (Para. 86, “As illustrated in FIG. 5A, the position pattern 501 includes at least one located pattern a, b or c. At least two of the located patterns a, b, and c correspond to different locations at a displayed illustration of the device under test, and have different areas.”).
Regarding claim 5, Huang as modified above teaches all of the elements of claim 4, as stated above, as well as wherein the first portion of the captured image receives IR illumination from the dot projector of the mobile device (Radiant; Pg. 5, “The use of image-based NIR measurement systems (for example, a sensor-based radiometric camera) for NIR source measurement can limit this complexity by capturing and measuring all emission points produced by a DOE across a large spatial area.” ).
Regarding claim 6, Huang as modified above teaches all of the elements of claim 5, as stated above, as well as wherein the step of comparing the captured image to the predetermined image includes comparing a second portion of the captured image with a corresponding second portion of the predetermined test image (Para. 86, “As illustrated in FIG. 5A, the position pattern 501 includes at least one located pattern a, b or c. At least two of the located patterns a, b, and c correspond to different locations at a displayed illustration of the device under test, and have different areas.”).
Regarding claim 7, Huang as modified above teaches all of the elements of claim 5, as stated above, as well as wherein the second portion of the captured image receives IR illumination from the IR camera of the mobile device (Radiant; Pg. 8, “As described above, some facial recognition systems rely on a “flood” function—a strong flash of NIR light used to detect a user’s face and determine focus distance. Like all NIR emissions, this flood function must also be tested to ensure it adheres to defined performance parameters.”).
Claim 9 corresponds to claim 1 and is rejected under the same analysis (See Huang Figs. 2-3, Paras. 5 and 34 for apparatus implementation details).
Claim 11 corresponds to claim 3 and is rejected under the same analysis.
Claim 12 corresponds to claim 4 and is rejected under the same analysis.
Claim 13 corresponds to claim 5 and is rejected under the same analysis.
Claim 14 corresponds to claim 6 and is rejected under the same analysis.
Claim 15 corresponds to claim 7 and is rejected under the same analysis.
Claim 17 corresponds to claim 1 and is rejected under the same analysis.
Claim 19 corresponds to claim 3 and is rejected under the same analysis.
Claim 20 corresponds to claim 4 and is rejected under the same analysis.
Claims 2, 10, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Huang as modified in view of Radiant above, further in view of Phryne (NPL, “Troubleshooting iPhone X Face ID Not Working 'Move iPhone a Little Lower/Higher'”, published 2021, pdf attached).
Regarding claim 2, Huang as modified in view of Radiant teaches all of the elements of claim 5, as stated above. They do not explicitly disclose wherein engaging the infrared ("IR") camera includes turning on a front camera of the mobile device and putting the front camera into portrait mode. However, Radiant necessarily engages the IR camera to test it.
Phryne teaches wherein engaging the infrared ("IR") camera includes turning on a front camera of the mobile device and putting the front camera into portrait mode (Pg. 4, “Let’s move on to the next step of Face ID troubleshooting – test the front camera parts. Open the camera app and swipe to Portrait Mode. The background is not blurred, which is abnormal. At the Portrait Mode, there is no flashing red light from the Dot Projector. We can confirm now that the Dot Projector has malfunctioned. This is the fault and the problem cannot be fixed.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Huang and Radiant to incorporate the teachings of Phryne to include wherein engaging the infrared ("IR") camera includes turning on a front camera of the mobile device and putting the front camera into portrait mode. Huang teaches an automated testing system for electronic devices including mobile phones, but does not explicitly disclose testing facial recognition components. Radiant teaches a method for testing facial recognition components, necessarily engaging them for testing, however they do not explicitly disclose putting the front camera into portrait mode to achieve this engagement. Phryne discloses the well-known technique of putting the front camera into portrait mode to engage the components of the facial recognition system. One of ordinary skill in the art would have recognized that in order to test components of a facial recognition system they have to be engaged. The technique disclosed by Phryne is a widely known method to engage the facial recognition system, leaving it obvious to implement in view of Huang’s disclosed “test mode”.
Claim 10 corresponds to claim 2 and is rejected under the same analysis.
Claim 18 corresponds to claim 2 and is rejected under the same analysis.
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Huang as modified in view of Radiant above, further in view of Lackey et al. (US Patent Pub. No. 2016/0364876 A1, published 2016).
Regarding claim 8, Huang as modified in view of Radiant teaches all of the elements of claim 1, as stated above, as well as comparing the captured image to the predetermined image. They do not explicitly disclose performing a luminance comparison.
Lackey discloses wherein the step of comparing the captured image to the predetermined image includes comparing a luminance value of a pixel or group of pixels of the captured image with a luminance value of a corresponding pixel or group of pixels of the predetermined test image (Para. 37, “In an exemplary embodiment, the difference between the average luminance value of each pixel partition of Image1 is compared to the average luminance value of the corresponding pixel partition of Image2.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Huang and Radiant to incorporate the teachings of Lackey to include wherein the step of comparing the captured image to the predetermined image includes comparing a luminance value of a pixel or group of pixels of the captured image with a luminance value of a corresponding pixel or group of pixels of the predetermined test image. Huang discloses a testing system which compares a captured test image to a predetermined image. Radiant discloses determining the intensity and uniformity of emissions from an IR camera. Neither explicitly disclose comparing luminance values. Lackey teaches a method for determining and comparing luminance values between two images to determine a correlation between them. One of ordinary skill in the art would have recognized that determining and comparing luminance values between two images is a well-known technique in the art to determine pixel level differences, as disclosed by Lackey. Including this processing technique in the modified system of Huang and Radiant would have predictably enhanced the accuracy of image comparison and provided a more robust analysis of the facial recognition components.
Claim 16 corresponds to claim 8 and is rejected under the same analysis.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A WAMBST whose telephone number is (703)756-1750. The examiner can normally be reached M-F 9-6:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ALEXANDER WAMBST/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights