DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 25 June 2026 has been entered.
Response to Arguments
Claims 1, 19, and 24 are amended. Claims 1-27 are pending in this action.
Applicant’s arguments, see pg. 9-12, filed 25 June 2026, with respect to the rejection(s) of claim(s) 1-27 under 35 U.S.C. 103 have been fully considered and are persuasive. Specifically, the applicant argues that the combination of Kim et al. (US 20220300591 A1; hereafter, Kim) with Kawamura and Rangesh Kawamura et al. (US 20220415085 A1; hereafter, Kawamura) in view of Rangesh et al. ("Take-over Time Prediction for Autonomous Driving in the Real-World: Robust Models, Data Augmentation, and Evaluation" full reference on the PTO-892 included with the previous action; hereafter, Rangesh) relies on an invalid motivation to combine non-analogous art. The examiner agrees. Kim is directed to an improvement in facial recognition technology and is not expressly used for augmenting training datasets making it non-obvious to combine Kim with Kawamura in view of Rangesh. The applicant further argues that Kawamura in view of Rangesh does not disclose determining a light condition of the accessed form of image data and further do not disclose processing the first artificial visual characteristic to adapt is to the light condition. The examiner agrees and stated so on pg. 7 of the action filed on 26 March 2026. Therefore, the rejection has been withdrawn. The applicant further argues that the order of adapting to the lighting condition prior to overlaying the characteristic on the image is significant because doing so in post processing requires significant computational overhead and introduces artifacts while adapting the artificial characteristic prior to overlay avoids these problems. The examiner agrees. Particularly, the examiner appreciates the evidence that the applicant found in Frolov et al. "Image Synthesis Pipeline for CNN-Based Sensing Systems." (full reference on PTO-892 included with the action filed 26 March 2026) showing the shortcomings of post processing and how pre processing may overcome those issues. Therefore, the rejection of claims 1-27 have been withdrawn.
However, upon further consideration and in light of the amendment to the independent claims, a new ground(s) of rejection is made in view of Suggu et al. (US 11702011 B1; hereafter, Suggu).
Suggu discloses:
determining a light condition of the accessed frame of image data (col. 32 line 27-30 "For example, a first brightness level is determined for the background image 1502B, and a second brightness level is determined for the second background image 1506.");
processing the first artificial visual characteristic to adapt to the determined light condition (col. 32 line 30-33 "An image property scale factor is determined based on the first and second brightness levels, and applied to scale the brightness level of the driver image 1502F") prior to superimposinq the first artificial visual characteristic onto the accessed frame of image data (col. 32 line 32-34 "applied to scale the brightness level of the driver image 1502F before the driver image 1502F is overlaid on the second background image 1506.");
The full rejection, including motivations to combine, is included below in the section "Claim Rejections - 35 USC § 103". Therefore, claims 1-27 are rejected under 35 U.S.C. 103.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 9-10, 17, 19-21, and 23-27 are rejected under 35 U.S.C. 103 as being unpatentable over Kawamura et al. (US 20220415085 A1; hereafter, Kawamura) in view of Rangesh et al. ("Take-over Time Prediction for Autonomous Driving in the Real-World: Robust Models, Data Augmentation, and Evaluation" full reference on the PTO-892 included with this action; hereafter, Rangesh) in further view of Suggu et al. (US 11702011 B1; hereafter, Suggu).
Regarding claim 1, Kawamura discloses:
A method for training a vehicular occupant monitoring system, the method comprising: accessing a frame of image data ([0027] the system acquires images from an acquisition device) captured by a camera ([0022] the acquisition device may be a camera) generating a first artificial visual characteristic for the occupant ([0028] a concealing image mask, which is understood as an artificial visual characteristic, is stored in a database. As it is being stored it must have been generated at a previous time);
generating a first modified frame of image data, wherein the first modified frame of image data comprises the accessed frame of the image data modified to include the adapted first artificial visual characteristic overlaying a first portion of the occupant ([0030] the concealing mask image, i.e. artificial characteristic, is super imposed on the original image to generate an image with the person in the image partially concealed. The adapted aspect is taught later in combination with Suggu);
generating a second artificial visual characteristic for the occupant ([0028] there are a plurality of concealing mask image objects, at least one of which is understood as a second artificial visual characteristic), wherein the second artificial visual characteristic is different than the adapted first artificial visual characteristic ([0028] the plurality of concealing mask images are a plurality of types, therefore, it is understood that the second concealing mask image is different than the first. The adapted aspect is taught later in combination with Suggu);
generating a second modified frame of image data, wherein the second modified frame of image data comprises the accessed frame of image data modified to include the second artificial visual characteristic overlaying a second portion of the occupant ([0030] "When there are a plurality of concealing mask images, the partially concealed image generation unit 103 generates, for each concealing mask image, concealed images in which a relevant concealing mask image is superposed on each of the face images." Based on the use of "each concealing mask image" and "each of the face images", it is understood that each face image, such as the first image, has a plurality of images generated related to it, one image with each one of the concealing mask images. At least one of these images is understood as the second modified frame of image data);
and training ([0049]-[0050] models are trained) using (i) the accessed frame of image data ([0049] a model is trained using the original images), (ii) the first modified frame of image data ([0050] a model is trained using the concealed image, i.e. the modified frame of image data) and (iii) the second modified frame of image data ([0050] as the model is trained using the plural "concealed images" it is understood to include at least the first and second modified frames of image data).
Kawamura is in the same field of endeavor of the instant application of preparing data for machine learning training with a concealing object, i.e. artificial visual characteristic, on an image of a person (Kawamura, [0023]).
Kawamura does not disclose expressly that the camera is disposed at a vehicle and viewing at least a portion of an occupant in the vehicle, and training the vehicular occupant monitoring system.
Rangesh discloses:
image data captured by a camera (pg. iii col. 2 para. 3, cameras are installed to capture data) disposed at a vehicle (pg. iii col. 2 para. 3, the cameras are installed in a vehicle) and viewing at least a portion of an occupant present in the vehicle (pg. iii col. 2 para. 3, the cameras view the occupant of the vehicle);
PNG
media_image1.png
120
348
media_image1.png
Greyscale
and training (pg. vi col. 2 para. 2, the LSTM model is trained)
PNG
media_image2.png
34
348
media_image2.png
Greyscale
the vehicular occupant monitoring system (pg. vi col. 1 para. 4, the model monitors the driver of the vehicle therefore is understood as a vehicular monitoring system)
PNG
media_image3.png
164
344
media_image3.png
Greyscale
Kawamura and Rangesh are combinable because Rangesh is in the related field of endeavor of augmenting data for training a machine learning model (Rangesh, pg. iv col. 1 para. 4).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the vehicle orientation of the camera and vehicle occupant monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura.
Kawamura in view of Rangesh does not disclose expressly to determine a light condition of the image and adapting the visual characteristic to the determined light condition.
Suggu discloses:
determining a light condition of the accessed frame of image data (col. 32 line 27-30 "For example, a first brightness level is determined for the background image 1502B, and a second brightness level is determined for the second background image 1506.");
processing the first artificial visual characteristic to adapt to the determined light condition (col. 32 line 30-33 "An image property scale factor is determined based on the first and second brightness levels, and applied to scale the brightness level of the driver image 1502F") prior to superimposing the first artificial visual characteristic onto the accessed frame of image data (col. 32 line 32-34 "applied to scale the brightness level of the driver image 1502F before the driver image 1502F is overlaid on the second background image 1506.");
Suggu is combinable with Kawamura in view of Rangesh because it is from the same field of endeavor of augmenting training data (Suggu, col. 1 line 15-19).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the light condition determination and visual characteristic adapting of Suggu with the invention of Kawamura in view of Rangesh.
The motivation for doing so would have been "the brightness level . . . [is] adjusted (624B) to ensure a realistic effect and fit into the first image" (Suggu, col. 19 line 4-7).
Therefore, it would have been obvious to combine Suggu with Kawamura in view of Rangesh to obtain the invention as specified in claim 1.
Regarding claim 2, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura further discloses:
The method of claim 1, wherein the first artificial visual characteristic comprises at least one selected from the group consisting of (i) a hat ([0028] the concealing object may be a hat), (ii) a beard and (iii) a tattoo (because of the wording “one selected from the group consisting of”, the examiner is interpreting this claim as a list of optional embodiments. Therefore, a teaching of one embodiment teaches the limitations of the claim).
Regarding claim 3, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura further discloses:
The method of claim 1, wherein the first artificial visual characteristic and the second artificial visual characteristic each comprise synthetic image data ([0030] the concealing mask image, i.e. artificial characteristic, is super imposed on the original image which is a synthetic modification of the original image. Therefore, the concealing mask image is understood as synthetic image data).
Regarding claim 4, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura further discloses:
The method of claim 1, wherein the first artificial visual characteristic and the second artificial visual characteristic do not overlay the eyes of the occupant ([0028] the concealing object may be a hat which does not overlay the eyes of the occupant).
Regarding claim 5, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura further discloses:
The method of claim 1, wherein training the vehicular occupant monitoring system comprises training a machine learning model ([0049]-[0050] machine learning models are trained)
Kawamura does not disclose expressly that the training of the machine learning model is of a machine learning model of a vehicular occupant monitoring system.
Rangesh discloses:
training a machine learning model (pg. vi col. 2 para. 2, the LSTM model is trained)
PNG
media_image2.png
34
348
media_image2.png
Greyscale
of the vehicular occupant monitoring system (pg. vi col. 1 para. 4, the model monitors the driver of the vehicle therefore is understood as a vehicular monitoring system).
PNG
media_image3.png
164
344
media_image3.png
Greyscale
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the machine learning model of the vehicle occupant monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura to obtain the invention as specified in claim 5.
Regarding claim 7, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura discloses:
The method of claim 1, wherein the first portion of the occupant comprises one selected from the group consisting of (i) hands of the occupant (because of the wording “one selected from the group consisting of”, the examiner is interpreting this claim as a list of optional embodiments. Therefore, a teaching of one embodiment teaches the limitations of the claim), (ii) hair of the occupant ([0030] relevant artificial characteristics are overlaid on the occupant. [0028] the artificial characteristic includes a hat which a person of ordinary skill in the art would understand to be associated with a hair portion of the occupant) and (iii) the face of the occupant ([0030] relevant artificial characteristics are overlaid on the occupant. [0028] the artificial characteristic includes sunglasses which a person of ordinary skill in the art would understand to be associated with a face portion of the occupant).
Regarding claim 9, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura further discloses:
The method of claim 1, wherein the first portion of the occupant and the second portion of the occupant are different ([0030] as "each" artificial characteristic is superimposed on "each" image, and as [0028] lists hats and sunglasses, it is understood that at least the hair portion and the face portion are used in superimposing which are different portions).
Regarding claim 10, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura does not disclose expressly that the image data is recorded using the camera while it is disposed at the vehicle.
Rangesh discloses:
The method of claim 1, wherein accessing the image data captured by the camera disposed at the vehicle comprises recording the image data using the camera while the camera is disposed at the vehicle (pg. iii col. 2 para. 3, the cameras are installed in a vehicle and record data while being disposed in the vehicle).
Regarding claim 17, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura does not disclose expressly wherein the occupant is the driver of the vehicle and the vehicular monitoring system comprises a driver monitoring system.
Rangesh discloses:
The method of claim 1, wherein the occupant of the vehicle is a driver of the vehicle (pg. iii col. 2 para. 3, the driver is the subject of the cameras, therefore the occupant is a driver of the vehicle) and the vehicular occupant monitoring system comprises a vehicular driver monitoring system (pg. iii col. 2 para. 3, the system monitors the driver during take-over request events which is understood as a vehicular driver monitoring system).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the driver monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura to obtain the invention as specified in claim 17.
Regarding claim 19, Kawamura discloses:
A method for training a vehicular occupant monitoring system, the method comprising: accessing a frame of image data ([0027] the system acquires images from an acquisition device) captured by a camera ([0022] the acquisition device may be a camera) generating a first artificial visual characteristic for the occupant ([0028] a concealing image mask, which is understood as an artificial visual characteristic, is stored in a database. As it is being stored it must have been generated at a previous time);
generating a first modified frame of image data, wherein the first modified frame of image data comprises the accessed frame of the image data modified to include the adapted first artificial visual characteristic overlaying a first portion of the occupant ([0030] the concealing mask image, i.e. artificial characteristic, is super imposed on the original image to generate an image with the person in the image partially concealed. The adapted aspect is taught later in combination with Suggu);
wherein at least one selected from the group consisting of (i) the first artificial visual characteristic comprises a hat ([0028] the concealing object may be a hat) and the first portion of the occupant comprises hair of the occupant ([0030] relevant artificial characteristics are overlaid on the occupant. [0028] the artificial characteristic includes a hat which a person of ordinary skill in the art would understand to be associated with a hair portion of the occupant), (ii) the first artificial visual characteristic comprises a beard and the first portion of the occupant comprises the face of the occupant and (iii) the first artificial visual characteristic comprises a tattoo and the first portion of the occupant comprises one selected from the group consisting of (a) hands of the occupant and (b) the face of the occupant (because of the wording “one selected from the group consisting of”, the examiner is interpreting this claim as a list of optional embodiments. Therefore, a teaching of one embodiment teaches the limitations of the claim);
generating a second artificial visual characteristic for the occupant ([0028] there are a plurality of concealing mask image objects, at least one of which is understood as a second artificial visual characteristic), wherein the second artificial visual characteristic is different than the adapted first artificial visual characteristic ([0028] the plurality of concealing mask images are a plurality of types, therefore, it is understood that the second concealing mask image is different than the first. The adapted aspect is taught later in combination with Suggu);
generating a second modified frame of image data, wherein the second modified frame of image data comprises the accessed frame of image data modified to include the second artificial visual characteristic overlaying a second portion of the occupant ([0030] "When there are a plurality of concealing mask images, the partially concealed image generation unit 103 generates, for each concealing mask image, concealed images in which a relevant concealing mask image is superposed on each of the face images." Based on the use of "each concealing mask image" and "each of the face images", it is understood that each face image, such as the first image, has a plurality of images generated related to it, one image with each one of the concealing mask images. At least one of these images is understood as the second modified frame of image data);
and training ([0049]-[0050] models are trained) using (i) the accessed frame of image data ([0049] a model is trained using the original images), (ii) the first modified frame of image data ([0050] a model is trained using the concealed image, i.e. the modified frame of image data) and (iii) the second modified frame of image data ([0050] as the model is trained using the plural "concealed images" it is understood to include at least the first and second modified frames of image data).
Kawamura is in the same field of endeavor of the instant application of preparing data for machine learning training with a concealing object, i.e. artificial visual characteristic, on an image of a person (Kawamura, [0023]).
Kawamura does not disclose expressly that the camera is disposed at a vehicle and viewing at least a portion of an occupant in the vehicle, and training the vehicular occupant monitoring system.
Rangesh discloses:
image data captured by a camera (pg. iii col. 2 para. 3, cameras are installed to capture data) disposed at a vehicle (pg. iii col. 2 para. 3, the cameras are installed in a vehicle) and viewing at least a portion of an occupant present in the vehicle (pg. iii col. 2 para. 3, the cameras view the occupant of the vehicle);
PNG
media_image4.png
130
382
media_image4.png
Greyscale
and training (pg. vi col. 2 para. 2, the LSTM model is trained)
PNG
media_image2.png
34
348
media_image2.png
Greyscale
the vehicular occupant monitoring system (pg. vi col. 1 para. 4, the model monitors the driver of the vehicle therefore is understood as a vehicular monitoring system)
PNG
media_image3.png
164
344
media_image3.png
Greyscale
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the vehicle orientation of the camera and vehicle occupant monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura.
Kawamura in view of Rangesh does not disclose expressly to determine a light condition of the image and adapting the visual characteristic to the determined light condition.
Suggu discloses:
determining a light condition of the accessed frame of image data (col. 32 line 27-30 "For example, a first brightness level is determined for the background image 1502B, and a second brightness level is determined for the second background image 1506.");
processing the first artificial visual characteristic to adapt to the determined light condition (col. 32 line 30-33 "An image property scale factor is determined based on the first and second brightness levels, and applied to scale the brightness level of the driver image 1502F") prior to superimposinq the first artificial visual characteristic onto the accessed frame of image data (col. 32 line 32-34 "applied to scale the brightness level of the driver image 1502F before the driver image 1502F is overlaid on the second background image 1506.");
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the light condition determination and visual characteristic adapting of Suggu with the invention of Kawamura in view of Rangesh.
The motivation for doing so would have been "the brightness level . . . [is] adjusted (624B) to ensure a realistic effect and fit into the first image" (Suggu, col. 19 line 4-7).
Therefore, it would have been obvious to combine Suggu with Kawamura in view of Rangesh to obtain the invention as specified in claim 19.
Regarding claim 20, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 19.
Kawamura further discloses:
The method of claim , wherein the first artificial visual characteristic and the second artificial visual characteristic each comprise synthetic image data ([0030] the concealing mask image, i.e. artificial characteristic, is super imposed on the original image which is a synthetic modification of the original image. Therefore, the concealing mask image is understood as synthetic image data).
Regarding claim 21, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 19.
Kawamura further discloses:
The method of claim 19, wherein training the vehicular occupant monitoring system comprises training a machine learning model ([0049]-[0050] machine learning models are trained)
Kawamura does not disclose expressly that the training of the machine learning model is of a machine learning model of a vehicular occupant monitoring system.
Rangesh discloses:
training a machine learning model (pg. vi col. 2 para. 2, the LSTM model is trained)
PNG
media_image2.png
34
348
media_image2.png
Greyscale
of the vehicular occupant monitoring system (pg. vi col. 1 para. 4, the model monitors the driver of the vehicle therefore is understood as a vehicular monitoring system).
PNG
media_image3.png
164
344
media_image3.png
Greyscale
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the machine learning model of the vehicle occupant monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura to obtain the invention as specified in claim 21.
Regarding claim 23, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 19.
Kawamura further discloses:
The method of claim 19, wherein the first portion of the occupant and the second portion of the occupant are different ([0030] as "each" artificial characteristic is superimposed on "each" image, and as [0028] lists hats and sunglasses, it is understood that at least the hair portion and the face portion are used in superimposing which are different portions).
Regarding claim 24, Kawamura discloses:
A method for training a vehicular occupant monitoring system, the method comprising: recording a frame of image data ([0027] the system acquires images from an acquisition device) using a camera ([0022] the acquisition device may be a camera) generating a first artificial visual characteristic for the occupant ([0028] a concealing image mask, which is understood as an artificial visual characteristic, is stored in a database. As it is being stored it must have been generated at a previous time);
generating a first modified frame of image data, wherein the first modified frame of image data comprises the recorded frame of the image data modified to include the adapted first artificial visual characteristic overlaying a first portion of the occupant ([0030] the concealing mask image, i.e. artificial characteristic, is super imposed on the original image to generate an image with the person in the image partially concealed. The adapted aspect is taught later in combination with Suggu);
generating a second artificial visual characteristic for the occupant ([0028] there are a plurality of concealing mask image objects, at least one of which is understood as a second artificial visual characteristic), wherein the second artificial visual characteristic is different than the adapted first artificial visual characteristic ([0030] as "each" artificial characteristic is superimposed on "each" image, and as [0028] lists hats and sunglasses, it is understood that at least the hair portion and the face portion are used in superimposing which are different portions. The adapted aspect is taught later in combination with Suggu), and wherein the first artificial visual characteristic and the second artificial visual characteristic each comprise synthetic image data ([0030] the concealing mask image, i.e. artificial characteristic, is super imposed on the original image which is a synthetic modification of the original image. Therefore, the concealing mask image is understood as synthetic image data);
generating a second modified frame of image data, wherein the second modified frame of image data comprises the recorded frame of image data modified to include the second artificial visual characteristic overlaying a second portion of the occupant ([0030] "When there are a plurality of concealing mask images, the partially concealed image generation unit 103 generates, for each concealing mask image, concealed images in which a relevant concealing mask image is superposed on each of the face images." Based on the use of "each concealing mask image" and "each of the face images", it is understood that each face image, such as the first image, has a plurality of images generated related to it, one image with each one of the concealing mask images. At least one of these images is understood as the second modified frame of image data);
and training ([0049]-[0050] models are trained) using (i) the recorded frame of image data ([0049] a model is trained using the original images), (ii) the first modified frame of image data ([0050] a model is trained using the concealed image, i.e. the modified frame of image data) and (iii) the second modified frame of image data ([0050] as the model is trained using the plural "concealed images" it is understood to include at least the first and second modified frames of image data).
Kawamura does not disclose expressly that the camera is disposed at a vehicle and viewing at least a portion of an occupant in the vehicle, and training the vehicular occupant monitoring system.
Rangesh discloses:
image data captured by a camera (pg. iii col. 2 para. 3, cameras are installed to capture data) disposed at a vehicle (pg. iii col. 2 para. 3, the cameras are installed in a vehicle) and viewing at least a portion of an occupant present in the vehicle (pg. iii col. 2 para. 3, the cameras view the occupant of the vehicle);
PNG
media_image4.png
130
382
media_image4.png
Greyscale
and training (pg. vi col. 2 para. 2, the LSTM model is trained)
PNG
media_image2.png
34
348
media_image2.png
Greyscale
the vehicular occupant monitoring system (pg. vi col. 1 para. 4, the model monitors the driver of the vehicle therefore is understood as a vehicular monitoring system)
PNG
media_image3.png
164
344
media_image3.png
Greyscale
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the vehicle orientation of the camera and vehicle occupant monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura.
Kawamura in view of Rangesh does not disclose expressly to determine a light condition of the image and adapting the visual characteristic to the determined light condition.
Suggu discloses:
determining a light condition of the accessed frame of image data (col. 32 line 27-30 "For example, a first brightness level is determined for the background image 1502B, and a second brightness level is determined for the second background image 1506.");
processing the first artificial visual characteristic to adapt to the determined light condition (col. 32 line 30-33 "An image property scale factor is determined based on the first and second brightness levels, and applied to scale the brightness level of the driver image 1502F") prior to superimposinq the first artificial visual characteristic onto the accessed frame of image data (col. 32 line 32-34 "applied to scale the brightness level of the driver image 1502F before the driver image 1502F is overlaid on the second background image 1506.");
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the light condition determination and visual characteristic adapting of Suggu with the invention of Kawamura in view of Rangesh.
The motivation for doing so would have been "the brightness level . . . [is] adjusted (624B) to ensure a realistic effect and fit into the first image" (Suggu, col. 19 line 4-7).
Therefore, it would have been obvious to combine Suggu with Kawamura in view of Rangesh to obtain the invention as specified in claim 24.
Regarding claim 25, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 24.
Kawamura further discloses:
The method of claim 24, wherein the first artificial visual characteristic comprises at least one selected from the group consisting of (i) a hat ([0028] the concealing object may be a hat), (ii) a beard and (iii) a tattoo (because of the wording “one selected from the group consisting of”, the examiner is interpreting this claim as a list of optional embodiments. Therefore, a teaching of one embodiment teaches the limitations of the claim).
Regarding claim 26, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 24.
Kawamura further discloses:
The method of claim 24, wherein the first artificial visual characteristic and the second artificial visual characteristic do not overlay the eyes of the occupant ([0028] the concealing object may be a hat which does not overlay the eyes of the occupant).
Regarding claim 27, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 24.
Kawamura further discloses:
The method of claim 24, wherein training the vehicular occupant monitoring system comprises training a machine learning model ([0049]-[0050] machine learning models are trained)
Kawamura does not disclose expressly that the training of the machine learning model is of a machine learning model of a vehicular occupant monitoring system.
Rangesh discloses:
training a machine learning model (pg. vi col. 2 para. 2, the LSTM model is trained)
PNG
media_image2.png
34
348
media_image2.png
Greyscale
of the vehicular occupant monitoring system (pg. vi col. 1 para. 4, the model monitors the driver of the vehicle therefore is understood as a vehicular monitoring system).
PNG
media_image3.png
164
344
media_image3.png
Greyscale
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the machine learning model of the vehicle occupant monitoring system of Rangesh with the invention of Kawamura.
The motivation for doing so would have been to "reliably predict takeover times for various secondary activities being performed by the drivers" (Rangesh, pg. ix col. 1 para. 2).
Therefore, it would have been obvious to combine Rangesh with Kawamura to obtain the invention as specified in claim 27.
Claims 6, 8, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Kawamura et al. (US 20220415085 A1; hereafter, Kawamura) in view of Rangesh et al. ("Take-over Time Prediction for Autonomous Driving in the Real-World: Robust Models, Data Augmentation, and Evaluation" full reference on the PTO-892 included with this action; hereafter, Rangesh) in further view of Suggu et al. (US 11702011 B1; hereafter, Suggu) and of Bowers et al. (US 9361447 B1; hereafter, Bowers).
Regarding claim 6, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly generating a third modified image with both the first and second artificial visual characteristic overlaying a respective portion of the occupant.
Bowers discloses:
The method of claim 1, further comprising generating third modified image data, wherein the third modified image data comprises the accessed frame of image data with the first artificial visual characteristic and the second artificial visual characteristic each overlaying (col. 9 line 23-27, multiple artificial visual characteristics may be overlaid on the same image) a respective portion of the occupant (col. 9 line 23-27, a person of ordinary skill in the art would understand that the hat would be overlaid the respective hair portion and a beard would be overlaid in a respective face portion in this example).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the image with more than one artificial visual characteristic of Bowers with the invention of Kawamura in view of Rangesh in further view of Suggu.
The motivation for doing so would have been "The process is illustratively configured so as to require a particular number of iterations and associated number of selected overlay effects that are sufficient to satisfy a specified minimum entropy measure" (Bowers, col. 6 line 28-31). In other words, selecting a number of overlay effects allows for greater variation or entropy. Greater variation would fulfill the suggestion of Rangesh to increase the number of samples to improve training (Rangesh, pg. ii col. 2 para. 2).
Therefore, it would have been obvious to combine Bowers with Kawamura in view of Rangesh in further view of Suggu to obtain the invention as specified in claim 6.
Regarding claim 8, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura in view of Rangesh in further view of Suggu does not disclose that the first and second portion are the same.
Bowers discloses:
The method of claim 1, wherein the first portion of the occupant and the second portion of the occupant are the same (col. 9 line 23-27, multiple artificial visual characteristics may be overlaid on the same image including a beard and sunglasses which would both be on the face portion of an occupant, therefore the first and second portion would be the same).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to make the first and second portion the same as in Bowers with the invention of Kawamura in view of Rangesh in further view of Suggu.
The motivation for doing so would have been "The process is illustratively configured so as to require a particular number of iterations and associated number of selected overlay effects that are sufficient to satisfy a specified minimum entropy measure" (Bowers, col. 6 line 28-31). In other words, selecting a number of overlay effects allows for greater variation or entropy. More options for variation become available when artificial visual characteristics may occupy the same portion. Greater variation would fulfill the suggestion of Rangesh to increase the number of samples to improve training (Rangesh, pg. ii col. 2 para. 2).
Therefore, it would have been obvious to combine Bowers with Kawamura in view of Rangesh in further view of Suggu to obtain the invention as specified in claim 8.
Regarding claim 22, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 19.
Kawamura in view of Rangesh in further view of Suggu does not disclose that the first and second portion are the same.
Bowers discloses:
The method of claim 19, wherein the first portion of the occupant and the second portion of the occupant are the same (col. 9 line 23-27, multiple artificial visual characteristics may be overlaid on the same image including a beard and sunglasses which would both be on the face portion of an occupant, therefore the first and second portion would be the same).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to make the first and second portion the same as in Bowers with the invention of Kawamura in view of Rangesh in further view of Suggu.
The motivation for doing so would have been "The process is illustratively configured so as to require a particular number of iterations and associated number of selected overlay effects that are sufficient to satisfy a specified minimum entropy measure" (Bowers, col. 6 line 28-31). In other words, selecting a number of overlay effects allows for greater variation or entropy. More options for variation become available when artificial visual characteristics may occupy the same portion. Greater variation would fulfill the suggestion of Rangesh to increase the number of samples to improve training (Rangesh, pg. ii col. 2 para. 2).
Therefore, it would have been obvious to combine Bowers with Kawamura in view of Rangesh in further view of Suggu to obtain the invention as specified in claim 22.
Claims 11-16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kawamura et al. (US 20220415085 A1; hereafter, Kawamura) in view of Rangesh et al. ("Take-over Time Prediction for Autonomous Driving in the Real-World: Robust Models, Data Augmentation, and Evaluation" full reference on the PTO-892 included with this action; hereafter, Rangesh) in further view of Suggu et al. (US 20220300591 A1; hereafter, Suggu) and of Peterson et al. (US 20210323473 A1; hereafter, Peterson).
Regarding claim 11, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly that the camera is dispose in the interior rearview mirror assembly of the vehicle.
Peterson discloses:
The method of claim 1, wherein the camera is disposed at an interior rearview mirror assembly of the vehicle ([0031] the camera is in the interior rearview mirror assembly).
Peterson is combinable with Kawamura in view of Rangesh in further view of Suggu because it is from the related field of endeavor of providing a system for driver monitoring (Peterson, [0004]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the rearview mirror disposed camera of Peterson with the invention of Kawamura in view of Rangesh in further view of Suggu.
The motivation for doing so would have been "an interior rearview mirror assembly that has a driver monitoring camera disposed at the mirror head so as to move in tandem with the mirror head when the mirror head is adjusted relative to an interior portion of the vehicle to adjust the driver's rearward view" (Peterson, [0004]).
Therefore, it would have been obvious to combine Peterson with Kawamura in view of Rangesh in further view of Suggu to obtain the invention as specified in claim 11.
Regarding claim 12, Kawamura in view of Rangesh in further view of Suggu and of Peterson discloses the subject matter of claim 11.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly that the camera in the rearview mirror assembly views through a mirror reflective element.
Peterson further discloses:
The method of claim 11, wherein the camera is disposed within a mirror head of the interior rearview mirror assembly of the vehicle ([0031] the camera is in the interior rearview mirror assembly), and wherein the camera views through a mirror reflective element of the mirror head of the interior rearview mirror assembly of the vehicle ([0031] the camera view through the mirror reflective element).
Regarding claim 13, Kawamura in view of Rangesh in further view of Suggu and of Peterson discloses the subject matter of claim 11.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly that the data captured by the camera is processed by an ECU and that the ECU is disposed at the interior of the rearview mirror assembly.
Peterson discloses:
The method of claim 11, wherein image data captured by the camera is processed by an ECU ([0035] the driver monitoring system (DMS) PCB receives data from the camera which is then communicated to the control PCB. [0031] the control PCB is understood as an ECU because it is an electronic control unit) and wherein the ECU is disposed at the interior rearview mirror assembly of the vehicle ([0031] and Fig. 4, the control PCB is within the mirror of the vehicle).
Regarding claim 14, Kawamura in view of Rangesh in further view of Suggu and of Peterson discloses the subject matter of claim 11.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly that the data captured by the camera is processed by an ECU and that the ECU is disposed remote from the mirror assembly.
Peterson discloses:
The method of claim 11, wherein image data captured by the camera is processed by an ECU ([0035] the driver monitoring system (DMS) PCB receives data from the camera which is then communicated to the control PCB. [0031] the control PCB is understood as an ECU because it is an electronic control unit), and wherein the ECU is disposed at the vehicle remote from the interior rearview mirror assembly (for the purpose of examination, the examiner interprets "disposed at the vehicle remote from" as any location in the vehicle which is not in the mirror. The examiner is not interpreting "the vehicle remote" as a single term describing a single object. [0052] the PCB may be disposed outside the mirror head and in the vehicle).
Regarding claim 15, Kawamura in view of Rangesh in further view of Suggu and of Peterson discloses the subject matter of claim 14.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly that the image data captured by the camera is transferred to the ECU by a coaxial cable.
Peterson discloses:
The method of claim 14, wherein image data captured by the camera is transferred to the ECU via a coaxial cable ([0060] connections between cameras and PCBs, i.e. ECU, may be coaxial cables).
Regarding claim 16, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly that the data captured by the camera is processed by an EDU and that the ECU can process at least one driving assist system of the vehicle.
Peterson discloses:
The method of claim 1, wherein image data captured by the camera is processed by an ECU ([0035] the driver monitoring system (DMS) PCB receives data from the camera which is then communicated to the control PCB. [0031] the control PCB is understood as an ECU because it is an electronic control unit), and wherein the ECU is operable to process the image data for at least one driving assist system of the vehicle (Claim 1, the system may process data from a forward viewing camera to identify hazards which is understood as a driving assist system. Claim 21, the second processor and the first processor of claim 1 may be the same processor, i.e. the ECU).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to include an ECU which processes a driving assist system as taught by Peterson in the invention of Kawamura in view of Rangesh in further view of Suggu.
The motivation for doing so would have been "to determine driving conditions and/or potential hazards ahead of the vehicle" (Peterson, [0004]).
Therefore, it would have been obvious to combine Peterson with Kawamura in view of Rangesh in further view of Suggu to obtain the invention as specified in claim 16.
Regarding claim 18, Kawamura in view of Rangesh in further view of Suggu discloses the subject matter of claim 1.
Kawamura in view of Rangesh in further view of Suggu does not disclose expressly wherein the occupant is the passenger of the vehicle and the vehicular monitoring system comprises an occupant detection system.
Peterson discloses:
The method of claim 1, wherein the occupant of the vehicle is a passenger of the vehicle ([0031] the system may view the passenger of the vehicle) and the vehicular occupant monitoring system comprises a vehicular occupant detection system ([0031] the system includes an occupant monitoring system which is understood as a detection system).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the occupant detection system of Peterson with the invention of Kawamura in view of Rangesh in further view of Suggu.
The motivation for doing so would have been to "provide occupant detection and/or monitoring functions" (Peterson, [0031]).
Therefore, it would have been obvious to combine Peterson with Kawamura in view of Rangesh in further view of Suggu to obtain the invention as specified in claim 18.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bhatia et al. (US 20240005641 A1) discloses a method and system for augmenting training data used to train model by replacing portions of a training image to create a new scenario for a model to be trained in.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA B CROCKETT whose telephone number is (571)270-7989. The examiner can normally be reached Monday-Thursday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA B. CROCKETT/Examiner, Art Unit 2661
/AARON W CARTER/Primary Examiner, Art Unit 2661
Prosecution Insights