Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification (MPEP 608.01, ¶6.31).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
First functional module, second functional module, third functional module, and EPD module
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Functional modules are considered to be part of a processor, microprocessor, microcontroller, DSP, ASIC, FPGA or discrete circuitry, as shown in Fig 1 and paragraph 0050.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
Claims 1-27 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of mathematical concepts without significantly more. The claims recite the abstract idea of receiving input data and calculating a confidence distribution, which corresponds to mathematical relationships and equations. This judicial exception is not integrated into a practical application and the claim(s) do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 1: Is the claim directed to a process, machine, manufacture, or composition of matter?
Yes, The claims are directed to process (claims 1-9) and a machine (claims 10-27).
Step 2A; is the claim directed to a law of nature, a natural phenomenon, or an abstract idea?
Yes, claims 1-27 are directed to the abstract idea of collecting, organizing and manipulating data. In essence, the independent claims recite; collection of data from a first and second module, and performing mathematical functions to calculate a confidence measure.
Prong One; Is the claim directed to a law of nature, a natural phenomenon, or an abstract idea?
Yes, as understood in their broadest reasonable interpretation, the independent claims are directed to the calculation of confidence using mathematical relationships to produce output data. Dependent claims further clarify the desired mathematical operations, such as using gaussian distributions and entropy. Additionally, determining whether to initiate one or more actions based on the results of a determination does not require active control of a component, as in the decision whether to, for example, play an alarm, is not a positive controlling step, as it merely involves whether it should be done, not controlling the vehicle/system to enact it.
Prong Two; Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the elements are generically recited. In particular, elements of a vehicle controller and sensors merely use generically recited features as tools to perform the abstract idea.
Functional Modules, which invoke generic processor programs, are recited at a high level of generality, and is recited to be applied onto generic processors or cores (such as in paragraph 0050 of the specification) that one of ordinary skill in the art would readily recognize.
Therefore, this abstract idea is not integrated into a practical application because there are no meaningful limits on practicing the abstract idea. Therefore, Claims 1-27 are directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Claims 1-27 do not include additional elements that amount to significantly more than the judicial exception. Dependent claims, such as claim 8-9, indicate potential interactions/sources of data, such as ADAS, Ad, DMS, or vision processes, but merely amount to extra solution activity as sources of information. For the same reasons as described above, with respect to integration of the abstract idea into a practical application, Claims 1-27 do not amount to significantly more than the judicial exception.
Using similar reasoning to above, Claims 1-27 do not add any significant structure or elements that qualify as significantly more, and instead merely further detail/define aspects of the abstract idea, and thus do not further integrate the abstract idea into a practical application.
Therefore, Claims 1-27 are not patent eligible under 35 U.S.C 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 6-11, 15-20, and 24-27 are rejected under 35 U.S.C. 103 as being unpatentable over Patel (US20200307586A1) in view of Sobhany (US20200239007A1).
Regarding claim 1, Patel teaches;
A method comprising:
performing data processing operations using multiple functional modules (taught as a cognitive processor, element 32, which includes modules to analyze data) to process input data from one or more types of driver sensors (taught as a sensor system, element 28, including sensing devices, element 40, which sense observable conditions on the exterior and interior of the vehicle, paragraph 0020), each functional module configured to perform one or more data processing operations in order to process input data and generate output data (taught as the modules being configured to receive an input, performing operations, and outputting data based on the performed operations, shown in Fig 2, paragraphs 0029-0033); and
for each functional module, generating a confidence measure associated with the output data generated by the functional module (taught as an evaluator module receiving confidence levels associated with hypothesis objects, paragraph 0033, which indicates calculation of a confidence level at the prior module);
wherein at least two of the functional modules are configured to operate logically sequentially (shown in the data flow in Fig 2 as a linear/sequential data path/pipeline) such that
a first of the functional modules provides the output data generated by the first functional module to a second of the functional modules (taught as the memory sending data to the hypothesizer module, paragraph 0038) and
the confidence measure associated with the output data generated by the second functional module is based at least partially on the confidence measure associated with the output data generated by the first functional module (taught as the evaluator module determining the entropy of distribution based on the input sensory/hypothesized data, paragraph 0033, where data is first received through the hypothesizer module and then sent to the evaluator module); and
wherein the multiple functional modules comprise heterogeneous functional modules configured to generate different types of output data (taught as the modules outputting different data, such as the decider module, which outputs environmental data regarding the most likely trajectories of each detected agent, paragraph 0028, while other modules output other data, such as the evaluator module receiving input environmental information and outputting contextual information including error and confidence measures, paragraph 0032).
However, Patel does not explicitly teach; estimate one or more driver characteristics based on the one or more types of driver sensors, and determine whether to initiate one or more actions based on the one or more estimated driver characteristics.
Sobhany teaches; estimate one or more driver characteristics based on the one or more types of driver sensors (taught as using vehicle sensors to detect driver parameters, such as expression, direction of the driver gaze, state of attentiveness etc. paragraph 0013) , and determine whether to initiate one or more actions based on the one or more estimated driver characteristics (taught as determining the driver attention, and causing an output by the vehicle based on the determination that, for example, the driver is not attentive, such as an alert, paragraph 0036)
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a driver attentiveness determination as taught by Sobhany in the system taught by Patel in order to improve autonomous handover and control. As suggested in Sobhany, one of the challenges in self-driving vehicles is requiring and determining that the human drivers continually pay attention to the environment/surroundings, even when in autonomous mode, which contributes to potential safety hazards when drivers do not pay attention or attempt to mitigate attention monitoring systems (paragraph 0002). One of ordinary skill in the art would recognize that verifying and monitoring attention, as described in Sobhany, would improve the overall system in Patel, which already implements several suggested sensors [cameras] directed to the interior of the vehicle, (paragraph 0020), to improve potential handoff/transitions between processing methods in a driving scenario.
Regarding claim 2, Patel as modified by Sobhany teaches;
The method of Claim 1 (see claim 1 rejection). Patel further teaches; wherein the confidence measures are based on entropy of predictive distribution (EPD) values determined using predictive distributions associated with the functional modules (taught as the evaluator determining the entropy of the distribution of confidence levels, paragraph 0033).
Regarding claim 6, Patel as modified by Sobhany teaches;
The method of Claim 1 (see claim 1 rejection). Patel further teaches; wherein:
a third of the functional modules provides the output data generated by the third functional module to the second functional module (taught as the evaluator module receiving data from hypothesis modules, paragraph 0033, where multiple hypothesizer modules may exist to evaluate data, paragraph 0040); and
the confidence measure associated with the output data generated by the second functional module is based at least partially on
the confidence measure associated with the output data generated by the first functional module (taught as the evaluator module computing contextual information regarding confidence levels of hypothesis objects, paragraph 0033) and
the confidence measure associated with the output data generated by the third functional module (taught as the evaluator module computing contextual information regarding confidence levels of hypothesis objects, paragraph 0033, and taking input from multiple hypothesizer modules, paragraph 0040; such that one hypothesizer module could correspond to a first functional module, and another hypothesizer module corresponds to a third functional module).
Regarding claim 7, Patel as modified by Sobhany teaches;
The method of Claim 1 (see claim 1 rejection). Patel further teaches; wherein:
each functional module includes or is associated with an entropy of predictive distribution (EPD) module (taught as calculating the entropy of distributions based on the evaluator module, paragraph 0033, indicating association between confidence levels and the evaluator); and
each EPD module is configured to determine the confidence measures for the output data generated by the associated functional module (shown as the evaluator module being associated/connected to other modules in the sequence, Fig 2).
Regarding claim 8, Patel as modified by Sobhany teaches;
The method of Claim 1 (see claim 1 rejection). However, Patel does not explicitly teach; wherein:
the one or more estimated driver characteristics comprise at least one of driver eyelid position, and a driver gaze direction.
Sobhany teaches; the one or more estimated driver characteristics comprise at least one [examiner interprets this to indicate only one of the following is required] of driver eyelid position, and a driver gaze direction (taught as detecting a direction of a driver’s gaze, and a rate of change of the direction of the driver’s gaze, in determining an overall driver attention, paragraph 0013).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a driver attentiveness determination as taught by Sobhany in the system taught by Patel in order to improve autonomous handover and control. As suggested in Sobhany, one of the challenges in self-driving vehicles is requiring and determining that the human drivers continually pay attention to the environment/surroundings, even when in autonomous mode, which contributes to potential safety hazards when drivers do not pay attention or attempt to mitigate attention monitoring systems (paragraph 0002). One of ordinary skill in the art would recognize that verifying and monitoring attention, as described in Sobhany, would improve the overall system in Patel, which already implements several suggested sensors [cameras] directed to the interior of the vehicle, (paragraph 0020), to improve potential handoff/transitions between processing methods in a driving scenario.
Regarding claim 9, Patel as modified by Sobhany teaches;
The method of Claim 1 (see claim 1 rejection). Patel further teaches; wherein the heterogeneous functional modules comprise
(i) at least one functional module configured to capture images of one or more scenes (taught as taking input of sensing devices of the environment, including camera data, paragraph 0020, using the data acquisition system, element 204) and
(ii) at least one functional module configured to process the images of the one or more scenes (taught as processing input data, such as a hypothesizer to predict/interpret data to determine potential trajectories of objects in the environment, such as by using the interface module, element 208).
Regarding claims 10-11, 15-20, and 24-27, it has been determined that no further limitations exist apart from those previously addressed in claims 1-2 and 6-9. Therefore, claims 10-11, 15-20, and 24-27 are rejected under the same rationales as claims 1-2 and 6-9, where claims 10-11 and 15-18 correspond to claims 1-2 and 6-9, and claims 19-20 and 24-27 correspond to claims 1-2 and 6-9 respectively.
Claim(s) 3, 11 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Patel (US20200307586A1) as modified by Sobhany (US20200239007A1), and further in view of Abdar (A review of uncertainty quantification in deep learning: techniques, applications and challenges, 2021, from the IDS).
Regarding claim 3, Patel as modified by Sobhany teaches;
The method of Claim 2 (see claim 2 rejection). However, Patel does not explicitly teach; wherein the predictive distributions associated with the functional modules comprise at least one of [examiner interprets this to indicate only one of the models are required]:
a predictive precision modeled using a univariate Gaussian distribution;
a predictive precision modeled using a multivariate Gaussian distribution; and
a predictive precision based on uncertainties associated with a machine learning model's parameters and uncertainties due to distributional mismatches between datasets associated with the machine learning model.
Abdar teaches; wherein the predictive distributions associated with the functional modules comprise at least one of [examiner interprets this to indicate only one of the models are required]:
a predictive precision modeled using a univariate Gaussian distribution (taught as using gaussian likelihood in determining a model likelihood, page 3 section 2.2, and using data-dependent uncertainty in form of precision, page 4 section 2.2);
a predictive precision modeled using a multivariate Gaussian distribution; and
a predictive precision based on uncertainties associated with a machine learning model's parameters and uncertainties due to distributional mismatches between datasets associated with the machine learning model.
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use a gaussian distribution as taught by Abdar in the system taught by Patel in order to improve predictions. As taught by Abdar, Gaussian likelihoods can be assumed for regression in representing the model precision (page 3 section 2.2).
Regarding claims 12 and 21, it has been determined that no further limitations exist apart from those previously addressed in claim 3. Therefore, claims 12 and 21 are rejected under the same rationales as claim 3.
Claim(s) 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Patel (US20200307586A1) as modified by Sobhany (US20200239007A1), and further in view of Cili (US20200053108A1).
Regarding claim 4, Patel as modified by Sobhany teaches;
The method of Claim 2 (see claim 2 rejection). However, Patel does not explicitly teach; wherein the EPD value associated with the output data generated by the second functional module is determined as a conditional entropy, the conditional entropy based on (i) the EPD value associated with the output data generated by the first functional module and (ii) a joint entropy.
Cili teaches; wherein the EPD value associated with the output data generated by the second functional module is determined as a conditional entropy, the conditional entropy based on (i) the EPD value associated with the output data generated by the first functional module and (ii) a joint entropy (taught as determining relationships for various information measures with correlated variables, including a joint entropy and a conditional entropy, paragraph 0047)
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement joint and conditional entropy considerations into a score as suggested by Cili in the system taught by Patel in order to improve certainty determinations. Such methods allow for determining mutual information between variables and determine importance of features (paragraph 0048). Patel already describes the combination of uncertainty from, for example, between sensors [relating to a joint entropy] (paragraph 0060) and a framework that determines a level of confidence [called complexity] to allow certain autonomous vehicle functions/control schemes (paragraph 0054). Thus, one of ordinary skill in the art would think to apply the more explicit techniques of calculating joint entropy to calculate a conditional entropy as suggested in Cili to determine a finalized complexity score as taught in Patel.
Regarding claim 5, Patel as modified by Sobhany and Cili teaches;
The method of Claim 4 (see claim 4 rejection). However, Patel does not explicitly teach; wherein the joint entropy is based on a joint probability associated with multiple discrete variables.
Cili teaches; wherein the joint entropy is based on a joint probability associated with multiple discrete variables (taught as a joint entropy being dependent on variables X and Y, paragraph 0047).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement joint and conditional entropy considerations into a score as suggested by Cili in the system taught by Patel in order to improve certainty determinations. Such methods allow for determining mutual information between variables and determine importance of features (paragraph 0048). Patel already describes the combination of uncertainty from, for example, between sensors [relating to a joint entropy] (paragraph 0060) and a framework that determines a level of confidence [called complexity] to allow certain autonomous vehicle functions/control schemes (paragraph 0054). Thus, one of ordinary skill in the art would think to apply the more explicit techniques of calculating joint entropy to calculate a conditional entropy as suggested in Cili to determine a finalized complexity score as taught in Patel.
Regarding claims 13-14 and 22-23, it has been determined that no further limitations exist apart from those previously addressed in claims 4-5. Therefore, claims 13-14 and 22-23 are rejected under the same rationales as claims 4-5, where claims 13-14 correspond to claims 4-5, and claims 22-23 correspond to claims 4-5 respectively.
Response to Arguments
The applicant argues on pages 16-17 of the remarks that the claims are eligible regarding 35 USC 101.
The examiner respectfully disagrees. The applicant’s arguments specifically address claim 10 (and its dependents) regarding the physical component aspects for addressing integration into a practical application. These presented physical components only include generic sensors and processors, without. Furthermore, the act of deciding whether to initiate an action is insufficient to integrate/positively control a system; as addressed above, merely making a decision does not indicate control of a component or system, especially beyond a simple extra-solution activity (such as displaying or producing an alert, which seems to be the primarily identified suggested response provided in the disclosure, e.g. warnings, paragraph 0017). Thus, the amendments to the claims do not sufficiently introduce components beyond generic components well known in the art, and do not integrate the recited idea into a practical application. As the claims continue to be directed to an abstract idea (gathering and processing data), and do not sufficiently introduce components or integrate into a practical application, the rejection is sustained.
Applicant argues on pages 17-19 that the amended claims are patentable over the recited prior art.
The examiner agrees that Patel does not anticipate the amended claims, and thus withdraws the previous 102 rejections. However, a new 103 rejection has been made in light of Sobhany to specifically address the use of sensors to identify driver gaze characteristics (e.g. paragraph 0013).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For further calculations of confidence in a model, pertaining to the independent claims; US20210319340A1
For further invocations of joint entropy, pertaining to claims 4-5, 13-14, and 22-23; US20130268465A1 and US20230229537A1
For detection of eyelids specifically in determining a driver characteristic, pertaining to, for example, claim 8; Bade US20200156649A1
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GABRIEL ANFINRUD whose telephone number is (571)270-3401. The examiner can normally be reached M-F 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at (571)270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GABRIEL ANFINRUD/Examiner, Art Unit 3662
/JELANI A SMITH/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights