DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim status
2. In response to the amendments filed 09/02/2026, claims 1 and 11 were amended and no claims were canceled and/or added. Therefore, claims 1-20 are currently pending for examination.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior office action.
Claim Interpretation
3. The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
4. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
5. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “emitting unit, configured to – corresponding to element 13 in Fig. 1” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
6. Claims 1, 7-11 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Raz (US 2024/0221396) in view of Lemley et al. (Lemley; US 2021/0398313).
For claim 1, Raz disclose a driver monitoring apparatus, comprising:
an emitting unit, configured to emit first light toward a driver [E.g. 0026: A first illumination source 20 is configured to emit a flood illumination 21 captured by the at least one imaging device 14 in the first image type 16 (FIG. 3). A second illumination source 22A is configured to emit an illumination pattern, such as a structured light illumination 24 captured by the at least one imaging device 14 in the second image type 18A (FIG. 3). A control system 100 includes at least one processor 104 that is configured to extract a 2-dimensional (“2D”) skeletal representation 26 of a vehicle occupant 28 from the first image type 16, measure a depth of the 2D skeletal representation 26 with the second image type 18A, and extrapolate a 3-dimensional (“3D”) skeletal representation 30 of the vehicle occupant 28 (FIGS. 4 and 5); 0003: monitoring system for a vehicle includes at least one imaging device configured to capture a first image type and a second image type in a sequence. A first illumination source is configured to emit a flood illumination captured by the at least one imaging device in the first image type. A second illumination source is configured to emit an illumination pattern captured by the at least one imaging device in the second image type; 0027];
a light sensor, configured to receive the first light reflected by the driver to generate first reflected light data [E.g. 0026: The monitoring system 10A includes at least one imaging device 14 (FIG. 2) configured to capture a first image type 16 and a second image type 18A in a sequence 19A (FIG. 3). A first illumination source 20 is configured to emit a flood illumination 21 captured by the at least one imaging device 14 in the first image type 16 (FIG. 3). A second illumination source 22A is configured to emit an illumination pattern, such as a structured light illumination 24 captured by the at least one imaging device 14 in the second image type 18A (FIG. 3), 0029: the first image type 16 includes 2D information about the occupant 28. The control system 100 (e.g., the at least one processor 104) may be configured to process the 2D information about the occupant 28 to detect locations within the first image type 16 that correspond to the body parts of interest 25A-H of the occupant 28, such as the limbs 25A, head 25B, neck 25C, joints 25D, hands 25E, fingers 25F, feet 25G, and torso 25H. The control system 100 (e.g., the at least one processor 104) may be configured to extract the 2D skeletal representation in accordance with the locations in the first image type 16 of the body parts of interest 25A-25H. The second image type 18A, on the other hand, includes depth information that can be overlaid on the 2D skeletal representation 26. More particularly, under the first mode of operation, the control system 100 (e.g., the at least one processor 104) may be configured to measure a depth of the 2D skeletal representation 26 with the depth information. The depth information may be obtained based on the principles of triangulation and known geometries between imaging device 14, the second illumination source 22A, and the distribution of the structured light illumination 24 (e.g., the light spot array)…, 0036, 0003-0004]; and
a processor [E.g. 0003: At least one processor is configured to extract a 2-dimensional (“2D”) skeletal representation of a vehicle occupant from the first image type, measure a depth of the 2D skeletal representation with the second image type, and extrapolate a 3-dimensional (“3D”) skeletal representation of the vehicle occupant. The at least one processor is further configured to generate at least one of a communication to the vehicle occupant to change a posture or a signal to a vehicle control system to move at least one of a position of a seat or a position of a steering wheel, 0004-0005,, 0026, 0030], coupled to the emitting unit and the light sensor, configured to execute the following operations:
calculating real-time depth data of the driver based on the first reflected light data [E.g. 0005: At least one processor is configured to extract a 2-dimensional (“2D”) skeletal representation of a vehicle occupant from the first image type, measure a depth of the 2D skeletal representation with the second image type, and extrapolate a 3-dimensional (“3D”) skeletal representation of the vehicle occupant, 0036: he control system 100 (e.g., the at least one processor 104) may be configured to extract the 2D skeletal representation in accordance with the locations in the first image type 16 of the body parts of interest 25A-25H. The second image type 18B, on the other hand, includes depth information that can be overlaid on the 2D skeletal representation 26. More particularly, under the second mode of operation, the control system 100 (e.g., the at least one processor 104) may be configured to measure a depth of the 2D skeletal representation 26 with the depth information. The depth information may be obtained based on the principles of a time difference between the emission of the beam illumination 63 in modulated pulses and the return of the beam illumination 63 back to the second imaging device 64, after being reflected from the vehicle occupant 28 (or other structure within the vehicle). The depth information may also be obtained by measuring the phase shift of the emission of the beam illumination 63 in continuous emission. In this manner, the first imaging device 14 and the second imaging device 64 may capture the first image type 16 and the second image type 18B simultaneously in a sequence 19B. It should be appreciated that, in some embodiments, the monitoring system 10B may not include the first illumination source 20 and the flood illumination 21 may be ambient lighting received from an environment. In some embodiments, the monitoring system 10B may further be configured to capture a 2D image of the interior cabin 38 (e.g., the occupant). For example, the first imaging device 14 and/or the second imaging device 64 may be configured to capture the 2D image. In this manner, the processor 104 may be configured to extract the 2D skeletal representation 26 from the 2D image rather than requiring additional sensors, 0004-0005];
comparing the real-time depth data and standard depth data to calculate a depth difference [E.g. 0039: Depth information about the 2D skeletal representation 26 can be obtained by comparing the first image type 16 and the second image type 18A-18C to extrapolate a 3D skeletal representation 30. For example, the depth extraction module 110 may include instructions for the processor 104 to determine the depth information on the basis of the principles of structured light (monitoring system 10A), ToF (monitoring system 10B), stereo vision (monitoring system 10C), or other depth calculating principles. Changes to the 3D skeletal representation 30 can be measured to obtain a present skeletal posture and movement of the 3D skeletal representation 30 in absolute scale. The 3D skeletal representation 30 may be monitored via the processor 104 by instructions contained in the drowsiness identifier module 112 or the orthopedic identifier module 114. Likewise, the absolute scale of the 3D skeletal representation 30 can be obtained within the interior cabin 38 or the exterior to the vehicle 12, 0040: the memory 106 may include machine learning algorithms, for example, deep learning, machine learning algorithms, tracking algorithms, and/or the like. More particularly, the control system 100 (e.g., the at least one processor 104) may be configured to modify parameters over continued usage when extracting the 2D skeletal representation 26 and extrapolate the 3D skeletal representation of the vehicle occupant 28. In some embodiments, for example, the vehicle occupant 28 may have a different baseline posture based on age, size, or medical condition such that the vehicle occupant 28 does not follow the generated communication or readjusts (steering wheel 31 and/or position of the seat 29) to a previous position after an automatic adjustment. In this manner, the control system 100 (e.g., the at least one processor 104) may accurately obtain information from occupants 28 with different posture parameters and adjust (or recommend an adjustment) the steering wheel 31 and/or position of the seat 29 accordingly. Similar methods may be applied to other functionalities of the monitoring system 10A-10C to improve accuracy and redefine parameters for occupants 28 of different sizes, shapes, medical conditions, and/or the like to improve ergonomics, posture, and attention, 0038, 0030]; and
determining whether to execute a warning operation based on the depth difference [E.g. 0003: The at least one processor is further configured to generate at least one of a communication to the vehicle occupant to change a posture or a signal to a vehicle control system to move at least one of a position of a seat or a position of a steering wheel, 0004-0005, 0026, 0031-0032].
Raz fails to expressly disclose recording the real-time depth data as one of a plurality of depth data updates; and adjusting the standard depth data by the one of the depth data updates.
However, as shown by Lemley, it was well known in the art of driver monitoring to include recording the real-time depth data as one of a plurality of depth data updates; and adjusting the standard depth data by the one of the depth data updates [E.g. 0055-0057, 0026, 0045; claim 12].
It would have been obvious to one of ordinary skill in the art of driver monitoring before the effective filling date of the claimed invention modify Raz with the teaching of Lemley in order to provide a more accurate way to monitor the driver by taking into consideration the depth updates and thereby enhance the overall accuracy of the monitoring system, also it is merely combining prior art elements according to known methods to yield predictable results.
For claim 7, Raz discloses wherein the standard depth data is generated by the following operations: emitting second light to the driver and receiving the second light reflected by the driver to generate second reflected light data, wherein the driver is in a standard driving posture; and calculating the standard depth data corresponding to the driver based on the second reflected light data [E.g. 0030-0031, 0040].
For claim 8, Raz discloses wherein the operation of determining whether to execute the warning operation further comprises: in response to the depth difference being greater than a difference threshold, calculating an abnormal duration; and in response to the abnormal duration exceeding a duration threshold, executing the warning operation [E.g. 0031, 0034, 0040].
For claim 9, Raz discloses wherein the warning operation comprises: adjusting a driving assistance parameter to increase a level of driving assistance involvement for a vehicle drove by the driver [E.g. 0031-0032, 0026].
For claim 10, Raz discloses wherein the warning operation comprises: generating a control signal to make an output apparatus to issue warning [E.g. 0026, 0031-0032].
For claim 11, is interpreted and rejected as discussed with respect to claim 1.
For claim 17, is interpreted and rejected as discussed with respect to claim 17.
For claim 18, is interpreted and rejected as discussed with respect to claim 18.
For claim 19, is interpreted and rejected as discussed with respect to claim 19.
For claim 20, is interpreted and rejected as discussed with respect to claim 20.
7. Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Raz in view of Lemley and further in view of Watanabe et al. (Watanabe; US 2022/0214435).
For claim 2, Raz discloses wherein the operation of calculating the real-time depth data further comprises:
transforming the first reflected light data to a plurality of first three-dimensional coordinates by using a time-of-flight measurement [E.g. 0026, 0029, 0036]; and
taking the first three-dimensional coordinates as the real-time depth data [E.g. 0026, 0029-0030, 0036, 0039-0040].
Raz in view of Lemley fails to expressly disclose the transforming of the reflected light happens by using a time-of-flight measurement.
However, as shown by Watanabe, it was well known in the art of measuring to include transforming of a reflected light by using a time-of-flight measurement [E.g. 0055-0057, Abstract, Figs. 6-7].
It would have been obvious to one of ordinary skill in the art of measuring before the effective filling date of the claimed invention modify Raz with the teaching of Watanabe in order to provide a more precise way to determine the 3D skeletal representation of the vehicle driver, and thereby enhance the overall accuracy of the monitoring system also it is merely combining prior art elements according to known methods to yield predictable results.
For claim 12, is interpreted and rejected as discussed with respect to claim 2.
Allowable Subject Matter
8. Claim 3-6 and 13-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Remarks
9. The Applicant's remarks regarding the rejection have been considered but are moot because the arguments do not apply to the new ground of rejection.
Conclusion
10. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
11. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMED BARAKAT whose telephone number is (571)270-3696. The examiner can normally be reached on 9:00am-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Davetta Goins can be reached on (571) 272-2957. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMED BARAKAT/
Primary Examiner, Art Unit 2689
