Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/27/2026 has been entered.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification (MPEP 608.01, ¶6.31).
Claim Objections
Claims 1, 10, and 19 are objected to because of the following informalities:
Claims 1, 10, and 19 recite “each confidence measure based upon…”. The examiner recommends amending this to “wherein each confidence measure is based upon…”.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
First functional module, second functional module, third functional module, and EPD module
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Functional modules are considered to be part of a processor, microprocessor, microcontroller, DSP, ASIC, FPGA or discrete circuitry, as shown in Fig 1 and paragraph 0050.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
The applicant’s amendment integrates the abstract idea into a practical application by actively controlling a vehicle actuator, and thus the 101 rejection is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3, 6-10, 12, 15-19, 21, and 25-27 are rejected under 35 U.S.C. 103 as being unpatentable over Arar (US20210182625A1) in view of Ahuja (US20200326667A1) and Malas (US20200074339A1).
Regarding claim 1, Arar teaches;
A driver monitoring method (taught as a distraction state network to estimate the distraction level of a vehicle driver, paragraph 0031) comprising:
performing data processing operations using multiple functional modules (shown in Fig 2, where data is passed through multiple modules including a facial landmark network, 220, and subsequently a drowsiness model, 250; wherein any number of network/models may be used, paragraph 0020) to process input data from one or more types of driver sensors (taught as receiving sensor data from one or more sensors, paragraph 0041, for example, cameras to monitor the state of the driver, paragraph 0053), estimate one or more driver characteristics based on the one or more types of driver sensors (taught as estimating states of a driver, such as drowsiness, paragraph 0028, cognitive load, paragraph 0030, and distraction, paragraph 0031), and operate one or more vehicle actuators to perform one or more vehicle actions based on the one or more estimated driver characteristics (taught as, upon determination of a distracted state, initiating operations such as an auto-pilot initiation, braking or turning operations or other actions, paragraph 0033) , each functional module configured to perform one or more data processing operations in order to process input data and generate output data (shown in Fig 2, where data is passed through multiple modules including a facial landmark network, 220, paragraph 0023, and subsequently a distraction/drowsiness model, 250, paragraph 0025); and
for [[each]] functional module, generating a confidence measure associated with the output data generated by the functional module (taught as, the models outputting a confidence score for facial landmarks, paragraph 0024, and using confidence values as inputs for drowsiness, paragraph 0028, and distraction, paragraph 0031; while this does not explicitly teach that each functional module performs a confidence scoring, at least a multiple number of them do, as any number of networks/models may be used in landmark detection, paragraph 0020),
wherein at least two of the functional modules are configured to operate logically sequentially (shown in Fig 2, where data is passed through multiple modules including a facial landmark network, 220, and subsequently a drowsiness model, 250) such that
a first of the functional modules provides the output data generated by the first functional module to a second of the functional modules (shown in Fig 2, where data is passed through multiple modules including a facial landmark network, 220, and subsequently a drowsiness model, 250, demonstrating a sequential process) and
wherein the multiple functional modules comprise heterogeneous functional modules configured to generate different types of output data of different dimensionalities (taught one network outputting two-dimensional positions of features of the subject’s face, paragraph 0025, and another network outputting a drowsiness state, such as a binary value, paragraph 0028; these indicate different dimensionalities where one is a 2d vector/matrix, plus confidence values, and another is a binary yes no, or a 1d output).
However, Arar does not explicitly teach; for each functional module, generating a confidence measure associated with the output data generated by the functional module each confidence measure based upon entropy of predictive distribution (EPD) values;
the confidence measure associated with the output data generated by the second functional module is based at least partially on the confidence measure associated with the output data generated by the first functional module;
and each confidence measure is determined as a scalar EPD value independent of a scale of variables of a predictive distribution associated with the corresponding functional module (emphasis added).
Ahuja teaches; for each functional module, generating a confidence measure associated with the output data generated by the functional module (taught as UE blocks being configured to calculate uncertainty estimates with respect to [each of] the sensors, paragraph 0027), each confidence measure based upon entropy of predictive distribution (EPD) values (taught as determining uncertainty estimates using predictive entropy, H(y*|x*, D), paragraphs 0123-0124);
and each confidence measure is determined as a scalar EPD value independent of a scale of variables of a predictive distribution associated with the corresponding functional module (indicated in the calculation of entropy, where the predictive entropy in equation 8 depends on probabilities and total number of output classes, neither of which should change from scale).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the entropy calculation as a measure of uncertainty/confidence as taught by Ahuja in the system taught by Arar in order to improve determination of uncertainty. As suggested by Ahuja, such a use of Entropy allows for capturing a combination of aleatoric and epistemic uncertainty in a single term (paragraph 0124, equation 8).
However, Ahuja does not explicitly teach; the confidence measure associated with the output data generated by the second functional module is based at least partially on the confidence measure associated with the output data generated by the first functional module.
Malas teaches; the confidence measure associated with the output data generated by the second functional module is based at least partially on the confidence measure associated with the output data generated by the first functional module (taught as determining respective information loss due to each link within a Markov chain [module, layers], paragraph 0083, being represented by entropy in equations 10a-10c)
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to ensure entropy is propagated in a system as suggested by Malas in the system taught by Arar in order to better account for uncertainty. As taught by Malas, the propagating effects of carious sources of uncertainty/entropy degrade the mutual information between the input and the output (paragraph 0039). Thus, it is important, from an information theory standpoint, to understand the loss of mutual information available between components in a systems level model to better account to information loss.
Regarding claim 3, Arar as modified by Ahuja and Malas teaches;
The method of Claim 1 (see claim 1 rejection). However, Arar does not explicitly teach; wherein the predictive distributions associated with the functional modules comprise at least one [examiner interprets this to indicate that only one of the following need be met] of:
a predictive precision modeled using a univariate Gaussian distribution;
a predictive precision modeled using a multivariate Gaussian distribution; and
a predictive precision based on uncertainties associated with a machine learning model's parameters and uncertainties due to distributional mismatches between datasets associated with the machine learning model.
Ahuja teaches; a predictive precision modeled using a univariate Gaussian distribution (taught as modeling layers within the multimodal fusion network with Gaussian distributions at each layer of the architecture, paragraph 0039, e.g. a Gaussian mixture model, paragraph 0062)
a predictive precision modeled using a multivariate Gaussian distribution (taught as implementing multivariate Gaussian models, paragraph 0062).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Gaussian models as taught by Ahuja in the system taught by Arar in order to improve modeling in a neural net architecture. As taught by Ahuja, using multivariate Gaussian models is advantageous as the features of hidden layers close to an output layer tend to be well represented by simpler Gaussian models, while features of hidden layers close to the input layer benefit from more precise modeling, for which Gaussian mixture models are a good fit (paragraph 0062).
Regarding claim 6, Arar as modified by Ahuja and Malas teaches;
The method of Claim 1 (see claim 1 rejection), wherein:
a third of the functional modules provides the output data generated by the third functional module to the second functional module (exemplified in Fig 2, with a facial detection module, a facial landmark network module, a gaze/head pose/drowsiness model, and a visualizer module in sequence. Furthermore, ant number of machine learning models or networks [modules] may be used that determine output quantities from landmark and confidence value inputs, paragraph 0020).
However, Arar does not explicitly teach; the confidence measure associated with the output data generated by the second functional module is based at least partially on (i) the confidence measure associated with the output data generated by the first functional module and (ii) the confidence measure associated with the output data generated by the third functional module.
Malas teaches; the confidence measure associated with the output data generated by the second functional module is based at least partially on (i) the confidence measure associated with the output data generated by the first functional module and (ii) the confidence measure associated with the output data generated by the third functional module (taught as determining respective information loss due to each link within a Markov chain [module, layers], paragraph 0083, being represented by entropy in equations 10a-10c; this propagation of entropy would include previous layers).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to ensure entropy is propagated in a system as suggested by Malas in the system taught by Arar in order to better account for uncertainty. As taught by Malas, the propagating effects of carious sources of uncertainty/entropy degrade the mutual information between the input and the output (paragraph 0039). Thus, it is important, from an information theory standpoint, to understand the loss of mutual information available between components in a systems level model to better account to information loss.
Regarding claim 7, Arar as modified by Ahuja and Malas teaches;
The method of Claim 1 (see claim 1 rejection). However, Arar does not explicitly teach; of wherein each functional module includes or is associated with an EPD module.
Ahuja teaches; wherein each functional module includes or is associated with an EPD module (taught as calculating the layer/module computes predictive entropy, paragraph 0124, and further suggests several uncertainty detection modules, 112 and 114 separate from UE modules).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the entropy calculation as a part of a module to be used as a measure of uncertainty/confidence as taught by Ahuja in the system taught by Arar in order to improve determination of uncertainty. As suggested by Ahuja, such a use of Entropy allows for capturing a combination of aleatoric and epistemic uncertainty in a single term (paragraph 0124, equation 8).
Regarding claim 8, Arar as modified by Ahuja and Malas teaches;
The method of Claim 1 (see claim 1 rejection), wherein the one or more estimated driver characteristics comprise at least one [examiner interprets this to indicate that only one of the following need be met] of driver eyelid position, and a driver gaze direction (taught as the camera capturing images of the eyes of the user, and computing gaze direction, paragraph 0034).
Regarding claim 9, Arar as modified by Ahuja and Malas teaches;
The method of Claim 1 (see claim 1 rejection). Arar further teaches; wherein the heterogeneous functional modules comprise (i) at least one functional module configured to capture images of one or more scenes (taught as a face detection [with input camera data that captures images, paragraph 0021], shown in Fig 2) and (ii) at least one functional module configured to process the images of the one or more scenes (taught as a face detection [with input camera data, paragraph 0021] and facial landmark network 220 [which processes the image to identify facial landmarks, paragraph 0023], shown in Fig 2).
Regarding claims 10, 12, 15-19, 21, and 24-27, it has been determined that no further limitations exist apart from those previously addressed in claims 1, 3, and 6-9. Therefore, claims 10, 12, 15-19, 21, and 24-27 are rejected under the same rationales as claims 1, 3, and 6-9, wherein;
Claims 10 and 19 correspond to claim 1,
Claims 12 and 21 correspond to claim 3,
Claims 15 and 24 correspond to claim 6,
Claims 16 and 25 correspond to claim 7,
Claims 17 and 26 correspond to claim 8, and
Claims 18 and 27 correspond to claim 9.
Claim(s) 4-5, 13-14, and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over Arar (US20210182625A1)as modified by Ahuja (US20200326667A1) and Malas (US20200074339A1) as applied to claim 1, and further in view of Cili (US20200053108A1).
Regarding claim 4, Arar as modified by Ahja and Malas teaches;
The method of Claim 1 (see claim 1 rejection). However, Arar does not explicitly teach; wherein the EPD value associated with the output data generated by the second functional module is determined as a conditional entropy, the conditional entropy based on (i) the EPD value associated with the output data generated by the first functional module and (ii) a joint entropy.
Cili teaches; wherein the EPD value associated with the output data generated by the second functional module is determined as a conditional entropy, the conditional entropy based on (i) the EPD value associated with the output data generated by the first functional module and (ii) a joint entropy (taught as determining relationships for various information measures with correlated variables, including a joint entropy and a conditional entropy, paragraph 0047).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement joint and conditional entropy considerations into a score as suggested by Cili in the system taught by Arar in order to improve certainty determinations. Such methods, as suggested by Cili, allow for determining mutual information between variables and determine importance of features (paragraph 0048).
Regarding claim 5, Arar as modified by Ahuja, Malas, and Cili teaches;
The method of Claim 4 (see claim 4 rejection). However, Arar does not explicitly teach; wherein the joint entropy is based on a joint probability associated with multiple discrete variables.
Cili teaches; wherein the joint entropy is based on a joint probability associated with multiple discrete variables (taught as a joint entropy being dependent on variables X and Y, paragraph 0047).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement joint and conditional entropy considerations into a score as suggested by Cili in the system taught by Patel in order to improve certainty determinations. Such methods, as suggested by Cili, allow for determining mutual information between variables and determine importance of features (paragraph 0048).
Regarding claims 13-14 and 22-23, it has been determined that no further limitations exist apart from those previously addressed in claims 4-5. Therefore, claims 13-14 and 22-23 are rejected under the same rationales as claims 4-5, where
Claims 13 and 22 correspond to claim 4, and
Claims 14 and 23 correspond to claim 5.
Response to Arguments
Applicant argues on pages 14-16 of the remarks that the claims re directed to eligible subject matter.
The examiner agrees, and withdraws the previous 101 rejection.
Applicant argues on pages 17-20 that the recited prior art does not sufficiently address the amended claim material.
The examiner agrees. However, a new rejection of the independent claims is made above, replacing the previously recited prior art with Arar, Ahuja, and Malas, which rectify the deficiencies identified in the arguments and the amendments.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For further calculations of confidence in a model, pertaining to the independent claims; US20210319340A1
For further invocations of joint entropy, pertaining to claims 4-5, 13-14, and 22-23; US20130268465A1 and US20230229537A1
For detection of eyelids specifically in determining a driver characteristic, pertaining to, for example, claim 8; Bade US20200156649A1
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GABRIEL ANFINRUD whose telephone number is (571)270-3401. The examiner can normally be reached M-F 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at (571)270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571, 3-10, 12-19, 21-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571, 3-10, 12-19, 21-272-1000.
/GABRIEL ANFINRUD/Examiner, Art Unit 3662
/JELANI A SMITH/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights