DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed on 2025 December 31 has been entered. The following is in reply to the Amendments and Arguments.
Claims amended: none
Claims cancelled: 3, 8, 12, 17, 23, 24
Claims added: none
Claims currently pending: 1, 2, 4-7, 9-11, 13-16, 18-22, 25, 26
Response to Arguments
Applicant, in the paragraph under the “REMARKS” heading, presents opening remarks regarding the disposition of the claims and the amendments to the claims. As no specific argument is raised in this/these section(s) with respect to the instant application, no rebuttal is required.
Applicant, in Section I.A, refers to independent claim 1 and argues that “Hayasaka does not perform masking to cover facial areas…and performs image recognition by scoring each of the facial areas that is not occluded”. This is a simplification of Hayasaka. While directed towards facial recognition where the subject’s face may be obscured by glasses or a mask, the invention disclosed therein detects a “shielding pattern” (i.e., a mask), the invention thereby performs the facial recognition on the areas (i.e., small regions) that are not comprised as being in the “shielding pattern”. Therefore, the occluded areas are masked from the facial recognition algorithm. Examiner notes that the entirety of the apparent support for this feature, as currently claimed, is found in Page 11, Line 30 – Page 12, Line 2:
In an embodiment, the system’s use of “RealFace” to perform facial image differentiation includes masking of features (including, in a non-limiting example, masking of the lower half of a human face), identification of micro-movements, and observation of gross and micro-movements indicating positional changes for the persons within the image to determine one unique human being from another.
The reader is left to speculate as to the use of masking in the RealFace algorithm to effect the determination from “one unique human being from another”. The disclosure of Hayasaka is directed towards a facial recognition algorithm that contains within it masking of portions of a human face, such as that covered by a mask. Therefore, Examiner readily concludes that the combination of references teaches a technique of facial recognition containing masking of the lower half of a human face.
Applicant, beginning in the final paragraph of page 8, argues, “While Hayasaka may be able to perform image recognition despite facial regions being masked, Hayasaka does not teach or suggest that its method is superior to a method where the entire face is available for analysis as is the case in Gast”. This argument is difficult to parse as a method of facial recognition that works even when facial regions are masked would be considered superior to a method that requires the entirety of a face in order to function. Applicant then argues that the combination of Gast in view of Hayasaka “would be non-sensical” because if a face is masked it would be “impossible to capture ‘the movements of the mouth and both the eyebrows in real-time’ as [in] Gast”. Assuming Applicant’s assertion that Gast is directed towards capturing the movements of the mouth and both the eyebrows in real-time in order to perform human facial tracking, the disclosure of Hayasaka would be beneficial to combine thereto in order to account for the cases where one or the other or both the mouth and eyebrows weren’t visible. Thus, the combination of the two references results in increase facial recognition, particularly in cases where only parts of a face are visible in an image. Therefore, Applicant’s arguments are unpersuasive and the grounds of rejection is herein maintained using the same references. Examiner notes that Applicant has NOT amended the claims in this response.
Applicant does not present any arguments in support of the patentability of the remaining claims except to rely on the arguments rebutted above and to assert that the claims are patentable based on their dependence from the independent claim(s). Therefore, said dependent claims stand rejected under the grounds of rejection presented herein and no detailed rebuttal is required.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 120 as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994)
The disclosure of the prior-filed application, Application No. 15/878,217, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application. The claims in the instant application, specifically claims 1, 2, 4-7, 9-11, 13-16, 18-22, 25, and 26 refer to counting the number of heads in a digital image and confirming the number of human faces based thereon. However, this content only finds support in Figures 5 and 6 and the description of said figures, which were added in the instant application and are not found in the parent application (i.e., 15/878,217).
References of Record but not Applied in the Current Grounds of Rejection
The prior art listed below is made of record as considered pertinent to applicant's disclosure and is not relied upon in the grounds of rejection presented in this Office action. Those starred with '*' were added to this list in this Office action. Those without "*" were added in a previous Office action and are not repeated on a PTO-892 Notice of References Cited form, but are maintained herein for informational purposes only.
Honary et al. (Pub. #: WO 2009/007752 A1) discloses a system for verification of occupancy of passenger vehicles that utilizes mobile devices to verify the occupancy of the vehicles for the purposes of toll collection.
Bala et al., in Chapter 13: Driver Monitoring from "Computer Vision and Imaging in Intelligent Transportation Systems", discusses a variety of techniques for facial recognition in the transportation industry.
Examiner's Note on the Format of the Prior Art Rejections
The prior art rejections below contain underlined markings of the limitations (e.g., sample limitation). The underlined portions of a claim are addressed at the end of the grounds of rejection for that claim. Examiner notes that the underlining of the claim language is not a statement that the primary reference does not teach that language, but simply that said claim language is addressed at the end of the grounds of rejection for that claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-6, 9-11, 13-15, 18-22, 25, 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Zafiroglu et al. (Pub. #: US 2014/0180773 A1) in view of
Gleeson-May et al. (Pub. #: US 2018/0012092 A1) in view of
Liao et al. (Pub. #: CN 112560557 A) in view of
"A Framework for Real-Time Face and Facial Feature Tracking Using Optical Flow Pre-Estimation and Template Tracking" by Gast in view of
Hayasaka (Pub. #: JP 2020038731 A).
Claims 1 and 10:
These claims are analogous with different representative embodiments; claim 1 is a method embodiment and claim 10 is a system embodiment. Zafiroglu teaches a computer system with computer-readable media in at least 0080 and 0081 for performing the steps:
A method of verifying commuter rewards based upon vehicle occupancy, comprising:
(Zafiroglu: Figure 3, 0011-0014, 0016, 0022)
capturing a digital image using a single mobile device, wherein the digital image comprises information indicating a number of occupants in a vehicle;
(Zafiroglu: 0029-0032)
obtaining GPS coordinates corresponding to a physical location of the mobile device;
determining whether the vehicle is eligible to access a high occupancy vehicle lane upon the information indicating the number of occupants in the vehicle and the GPS coordinates;
(Zafiroglu teaches verifying the occupancy reported by other sensors by utilizing an "imaging device to support or reject the occupancy indicated" in at least 0018 and 0028-0032. Zafiroglu teaches that his detection can be triggered at a particular location by teaching the use of the system in determining tolls via "checkpoints" in at least 0025-0027 and in response to pre-determined conditions in at least 0025. Zafiroglu discloses that HOV "lanes are legally available only to those vehicles having a number of occupants that meets or exceeds a threshold number" in at least 0013 and discloses not charging a fee to use an HOV lane if the number of occupants exceeds a threshold in at least 0015 which corresponds to determine eligibility to use the HOV lane.)
and the mobile device displaying an indication based upon the determining;
(Zafiroglu: 0040, presents occupancy information to the users in at least 0076)
counting multiple human heads in said digital image, and emplacing each human head within a box, where the boxed heads determines the number of occupants in the vehicle;
determining that the boxed heads depict real human faces, wherein determining that the boxed heads depict real human faces comprises performing masking of facial features, identification of micro-movements, and observation of gross and micro-movements indicating positional changes.
As for, "single mobile", "obtaining GPS coordinates corresponding to a physical location of the mobile device;", "mobile":
Zafiroglu teaches using the location information from a vehicle device for determining an incentive or reward amount in at least 0024-0025. Zafiroglu does not appear to make explicit the use of a mobile device to gather the information (i.e., the "digital image" and the "GPS coordinates corresponding to a physical location of the mobile device"). However, Gleeson-May teaches a technique of using a mobile device’s sensors to identify users in a vehicle in at least 0058, 0059, 0064, including identifying human faces in an image in at least 0067.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the occupancy/location based incentive system of Zafiroglu with the technique of using a mobile device to gather the data used to determine incentives as taught by Gleeson-May. Motivation to combine the references come from the desire to positively identify and reward a driver for performing in a desired fashion (Gleeson-May: 0024-0027).
As for, "counting multiple human heads in said digital image, and emplacing each human head within a box, where the boxed heads determines the number of occupants in the vehicle;"
Zafiroglu does not appear to specify counting the number of human heads in an image and confirming human faces based on the number of heads. However, Liao teaches a system that captures an image in a vehicle such as a bus in at least the second paragraph on page 17 and that uses a head detection algorithm to identify and count heads in the image as taught in the last two paragraphs of page 15 and the first paragraph of page 16. Liao further teaches the use of a face detection method that depends upon the human head detection in the final 3 complete paragraphs of page 19 and in Figure 7.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the rewards based on a detected occupancy of a vehicle using images system of Zafiroglu with the number detection and face verification technique of Liao. Motivation to combine the two references comes from the desire to lower the detection cost and increase the detection precision of a person detection system (Liao: second paragraph of page 15).
As for, "determining that the boxed heads depict real human faces, wherein determining that the boxed heads depict real human faces comprises performing masking of facial features, identification of micro-movements, and observation of gross and micro-movements indicating positional changes."
Zafiroglu in view of Gleeson-May in view of Liao does not appear to specify performing the "determining that the boxed heads depict real human faces" via "performing masking of facial features, identification of micro-movements, and observation of gross and micro-movements indicating positional changes". However, Gast teaches a method of facial recognition that includes tracking facial features and head pose in at least section 2.2, including masking of facial features in at least sections 4.3 and 5.2 which is evident from the masking of the hair, ears, and neck from the facial tracker sub-algorithms for tracking of the mouth and eyebrows.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the rewards based on a detected occupancy of a vehicle using images system of Zafiroglu in view Gleeson-May in view of Liao of the number detection and face verification technique of Liao with the further techniques of facial recognition/tracking of Gast. Motivation to combine Zafiroglu in view of Gleeson-May in view of Liao with Gast comes from all references pertaining to facial recognition techniques and in order to "deal with problems such as illumination changes, occlusions, pose changes, fast movement etc." (Gast, Section 1 "Introduction").
Zafiroglu in view of Gleeson-May in view of Liao in view of Gast discloses a system that utilizes face recognition techniques to determine vehicle occupancy for the determination of rewards/incentives. The combination does not appear to make explicit the use of “masking of facial features” as claimed. However, Hayasaka teaches a technique of facial recognition that contains a “shielding pattern” (i.e., a masking pattern) that comprises the “lower half area of the face” in at least the first 5 paragraphs of page 3.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the facial recognition subsystem of the above identified combination of references with the technique of a shielding pattern as taught by Hayasaka. Motivation to combine the combination of references with Hayasaka comes from the use of facial recognition techniques and to improve the “recognition accuracy for facial images with covered areas without increasing system building cost and recognition processing load” (Hayasaka: Abstract).
Claims 2 and 11:
where said digital image is either a photograph or an image obtained from said image capture device.
(Zafiroglu: 0029-0032)
Claims 4 and 13:
where the boxed head count requires at least one of a calculation, determination, or analysis of a facial signature.
Zafiroglu does not appear to specify counting the number of human heads in an image without computing a facial signature. However, Liao teaches a system that captures an image in an elevator that uses a head detection algorithm to identify and count heads in the image as taught in the last two paragraphs of page 15 and the first paragraph of page 16. Liao further teaches the use of a face detection method that depends upon the human head detection in the final 3 complete paragraphs of page 19 and in Figure 7.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the rewards based on a detected occupancy of a vehicle using images system of Zafiroglu with the number detection and face verification technique of Liao. Motivation to combine the two references comes from the desire to lower the detection cost and increase the detection precision of a person detection system (Liao: second paragraph of page 15).
Claims 5 and 14:
wherein the determining of whether the vehicle is eligible to access the high occupancy vehicle lane is based at least in part on at least the vehicle, driver, or riders meeting a pre-set threshold condition.
(Zafiroglu: "[0015] In certain embodiments, the incentives provided based on increased vehicle occupancy may be further based on one or more additional parameters relating to operation of a vehicle such as, the example, a time of day during which a vehicle is operated. As a non-limiting example, an increased incentive may be provided based on increased vehicle occupancy during a peak travel period as compared to a non-peak travel period. In this manner, vehicle operators may be incentivized to increase vehicle occupancy during peak travel periods. Further, in certain embodiments, the incentives may include an elimination of a fee altogether or an incentive payment. Referring again to the above non-limiting example, if a vehicle is determined to have a threshold number of occupants ( e.g., three or more) during a peak travel period, a fee ( e.g., a toll amount) typically associated with operation of the vehicle during that time period may be eliminated or an incentive payment may be made to an appropriate entity associated with the vehicle ( e.g., registered owner of the vehicle).", 0019, 0020, 0025, 0046, 0060)
Claims 6 and 15:
further comprising verifying the count of the multiple human heads where said multiple human heads are verified through facial image differentiation.
Zafiroglu does not appear to specify multiple human faces verified by facial image differentiation. Liao further teaches the use of a face detection method that depends upon the human head detection in the final 3 complete paragraphs of page 19 and in Figure 7.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the rewards based on a detected occupancy of a vehicle using images system of Zafiroglu with the number detection and face verification technique of Liao. Motivation to combine the two references comes from the desire to lower the detection cost and increase the detection precision of a person detection system (Liao: second paragraph of page 15).
Claim 9:
Zafiroglu does not appear to specify counting the number of heads using a facial recognition signature. However, Gleeson-May teaches a two pass facial recognition system that first uses an algorithm to identify faces in an image in 0066 including "a projected area of a head region" which are then subsequently "extracting an image segment containing the detected face as the biometric signal" in at least 0067, with the biometric signal then "processed into a user identifier" in at least 0068.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the rewards based on a detected occupancy of a vehicle using images system of Zafiroglu with the two-stage facial recognition technique based on identifying head regions using a facial signature as taught by Gleeson-May. Motivation to combine the references come from the desire to positively identify and reward a driver for performing in a desired fashion (Gleeson-May: 0024-0027).
Claim 18:
where the single image capture device is a camera.
(Zafiroglu: 0029-0032, 0037, 0040)
Claims 19 and 20:
said mobile device transmitting the information indicating the number of occupants in the vehicle and the GPS coordinates of the vehicle.
Zafiroglu teaches using the location information from a vehicle device for determining an incentive or reward amount in at least 0024-0025. Zafiroglu does not appear to make explicit the use of a mobile device to gather the information (i.e., the "digital image" and the "GPS coordinates corresponding to a physical location of the mobile device"). However, Gleeson-May teaches a technique of using a mobile device’s sensors to identify users in a vehicle in at least 0058, 0059, 0064, including identifying human faces in an image in at least 0067.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the occupancy/location based incentive system of Zafiroglu with the technique of using a mobile device to gather the data used to determine incentives as taught by Gleeson-May. Motivation to combine the references come from the desire to positively identify and reward a driver for performing in a desired fashion (Gleeson-May: 0024-0027).
Claims 21 and 22:
wherein the determining of whether the vehicle is eligible to access the high occupancy vehicle lane is performed at a validation point positioned such that the determining is performed immediately prior to the vehicle accessing the high occupancy vehicle lane.
(Zafiroglu discloses "tolling infrastructure" in at least 0034-0035 which includes a toll gate as displayed in Figure 1B that is a "checkpoint" at which tolls are calculated. The tolling infrastructure include an "RFID interrogator" that communicates with vehicles "within a suitable range" in at least 0051.)
Claims 25 and 26:
wherein the masking of facial features comprises masking of the lower half of a human face.
Zafiroglu in view of Gleeson-May in view of Liao in view of Gast discloses a system that utilizes face recognition techniques to determine vehicle occupancy for the determination of rewards/incentives. The combination does not appear to make explicit the use of “masking of facial features” as claimed. However, Hayasaka teaches a technique of facial recognition that contains a “shielding pattern” (i.e., a masking pattern) that comprises the “lower half are of the face” in at least the first 5 paragraphs of page 3.
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the facial recognition subsystem of the above identified combination of references with the technique of a shielding pattern as taught by Hayasaka. Motivation to combine the combination of references with Hayasaka comes from the use of facial recognition techniques and to improve the “recognition accuracy for facial images with covered areas without increasing system building cost and recognition processing load” (Hayasaka: Abstract).
Claims 7, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over:
Zafiroglu et al. (Pub. #: US 2014/0180773 A1) in view of
Gleeson-May et al. (Pub. #: US 2018/0012092 A1) in view of
Liao et al. (Pub. #: CN 112560557 A) in view of
"A Framework for Real-Time Face and Facial Feature Tracking Using Optical Flow Pre-Estimation and Template Tracking" by Gast in view of
Hayasaka (Pub. #: JP 2020038731 A) in view of
Glazer et al. (Pub. #: US 2019/0114488 A1).
Claims 7 and 16:
determining a need for verification of the count of the multiple human heads is based at least in part on a function of a historical compliance with rules regarding high occupancy lane access of a said mobile device user as is indicated by a trust score.
Zafiroglu in view of Liao teaches verifying the number of human faces. Zafiroglu does not appear to specify using a trust score based on historical compliance to determine if a verification function needs to be performed. However, Glazer teaches a system for verification in vision-based systems that includes a technique of evaluating a user to determine a likelihood of accuracy in using the system in at least 0070-0074 that is used to determine if additional checks should be performed on the user’s use of the system or if the user should be trusted in at least 0108 with the teaching of a user’s “history of monitoring alignment” (see also 0103-0108).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the system of rewarding a user for car-pooling based on sensor-based presence detection with the technique of reducing the frequency of verification checks based on a user's history of compliance with the system as taught by Glazer. Motivation to combine the references come from the desire to verify the results of computer vision systems which expose systems to “a wide range of attack surfaces” (Glazer: 0003, 0009, 0010).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT SNIDER whose telephone number is (571)272-9604. The examiner can normally be reached M-W: 9:00-4:30 Mountain (11:00-6:30 Eastern).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Waseem Ashraf can be reached at (571)270-3948. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.S/Examiner, Art Unit 3621
/WASEEM ASHRAF/Supervisory Patent Examiner, Art Unit 3621
Prosecution Insights