DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendment filed 06/18/2026 has been entered. Claims 1-20 remain pending.
Response to Arguments
Applicant’s arguments, see Pages 15-20, filed 06/18/2026, with respect to the 35 U.S.C. 101 rejection of Claims 1-20 have been fully considered and are persuasive. The 35 U.S.C. 101 rejection of Claims 1-20 has been withdrawn.
Applicant’s arguments, see Pages 20-26, filed 06/18/2026, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection of Claims 1, 3, and 5-8 are made under 35 U.S.C. 102 in view of newly discovered prior art Kryst (DE102021201136A1), and Claims 2, 6, 9, and 10-20 are made under 35 U.S.C. 103 in view of newly discovered prior art Kryst (DE102021201136A1) in view of previously disclosed prior arts Rangel (US20190234920), Purdy (US20230058731), and Heidi (US20160343136).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-9 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claims 1-4 and 8 detail an “adjustable mechanical assembly of the vehicle”. The specification details in [0040] an adjustable component. The specification are silent with regards to an adjustable mechanical assembly of the vehicle.
Claims 2-9 are rejected due to dependence on Claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, and 5-8 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kryst (DE102021201136A1).
In regards to Claim 1, Kryst teaches “determining, based at least on first sensor data obtained using one or more first sensors of a vehicle, a location associated with an adjustable mechanical assembly of the vehicle which is located within an interior of the vehicle (The invention relates to a method and a device for monitoring the interior of a vehicle and a vehicle comprising a corresponding device according to the independent claims - [0001]; Advantageously, the method includes an evaluation of measurement data [i.e. first sensor data] from at least one sensor [i.e. one or more first sensors] to detect the position of the rearview mirror [i.e. an adjustable mechanical assembly]. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]; The at least one sensor can be a magnetic angle sensor and/or an accelerometer and/or a force sensor and/or a gyroscope and/or a vibration sensor. A magnetic angle sensor can be arranged, in particular, between the rearview mirror and a foot of the rearview mirror. The aforementioned force sensor can, in particular, be located at one of the bases of the rearview mirror. The gyroscope and/or the vibration sensor and/or the accelerometer may also be located on the foot and/or on the rearview mirror. A magnetic angle sensor and/or an accelerometer have the advantage that both a change in the position of the rearview mirror and the exact deviation from the basic position can be detected - [0011]);
determining, based at least on the location associated with the adjustable mechanical assembly, to update one or more first values for one or more calibration parameters of a second sensor attached to the adjustable mechanical assembly to include one or more second values for the one or more calibration parameters (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. second sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. second sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]); and
causing, based at least on second sensor data obtained using the second sensor and the one or more second values of the one or more calibration parameters, performance of one or more operations associated with the vehicle (Based on the calibration of the mirror position, an error-free assignment of the driver for driver observation is possible [i.e. performance of one or more operations associated with the vehicle]. It is also possible to create an optimal image for video telephony, so that the recorded images can be optically aligned in such a way that they are independent of the position of the rearview mirror. Furthermore, a relationship to the instrument panel can be determined for distance detection between the occupants and an airbag, and individual control of the restraint systems can be achieved. – [0012])”
In regards to Claim 3, Kryst discloses the claimed invention as detailed above. Kryst further teaches “determining, based at least on a previous location associated with the adjustable mechanical assembly, the one or more first values for the one or more calibration parameters of the second sensor, wherein the determining to update the one or more first values for the one or more calibration parameters to include the one or more second values for the one or more calibration parameters is further based at least on the one or more first values for the one or more calibration parameters (The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]).”
In regards to Claim 5, Kryst discloses the claimed invention as detailed above. Kryst further teaches “receiving third sensor data obtained using the second sensor, wherein the determining the to update one or more first values for the one or more calibration parameters to the one or more second values for the one or more calibration parameters is further based at least on the third sensor data (Calibration can be performed in particular using image recognition. For this purpose, the camera is adjusted to a specific image section before delivery of the vehicle, whereby the image section includes one or more reference points, with the help of which a positional deviation of the rearview mirror can be calculated. Possible reference points could be, for example, the rear wiper and/or the top edge of the sky and/or window cutouts. In particular, appropriate calibration can be carried out at regular intervals and/or directly before an action is to be performed by the vehicle - [0009]).”
In regards to Claim 6, Kryst discloses the claimed invention as detailed above. Kryst further teaches “receiving third sensor data obtained using the second sensor, the third sensor data representative of a sensor representation; and determining that a portion of the sensor representation depicts a feature located within the vehicle, wherein the determining to update the one or more first values for the one or more calibration parameters to the one or more second values for the one or more calibration parameters is further based at least on the determining that the portion of the sensor representation depicts the feature (Calibration can be performed in particular using image recognition. For this purpose, the camera is adjusted to a specific image section before delivery of the vehicle, whereby the image section includes one or more reference points [i.e. sensor representation], with the help of which a positional deviation of the rearview mirror can be calculated. Possible reference points could be, for example, the rear wiper and/or the top edge of the sky and/or window cutouts. In particular, appropriate calibration can be carried out at regular intervals and/or directly before an action is to be performed by the vehicle - [0009]).”
In regards to Claim 7, Kryst discloses the claimed invention as detailed above. Kryst further teaches “determining, based at least on the second sensor data and the one or more second values for the one or more calibration parameters, a location of an object within the interior of the vehicle, wherein the causing the performance of the one or more operations associated with the vehicle is based at least on the location of the object within the interior of the vehicle (Based on the calibration of the mirror position, an error-free assignment of the driver for driver observation is possible [i.e. performance of one or more operations associated with the vehicle]. It is also possible to create an optimal image for video telephony, so that the recorded images can be optically aligned in such a way that they are independent of the position of the rearview mirror. Furthermore, a relationship to the instrument panel can be determined for distance detection between the occupants and an airbag, and individual control of the restraint systems can be achieved. – [0012]).”
In regards to Claim 8, Kryst discloses the claimed invention as detailed above. Kryst further teaches “receiving one or more inputs associated with an adjustment to a position of the adjustable mechanical assembly within the interior of the vehicle (The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]); and receiving, after the adjustment to the position of the adjustable mechanical assembly, the first sensor data generated obtained using the one or more first sensors (Based on the calibration of the mirror position, an error-free assignment of the driver for driver observation is possible [i.e. performance of one or more operations associated with the vehicle]. It is also possible to create an optimal image for video telephony, so that the recorded images can be optically aligned in such a way that they are independent of the position of the rearview mirror. Furthermore, a relationship to the instrument panel can be determined for distance detection between the occupants and an airbag, and individual control of the restraint systems can be achieved. – [0012]).”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Kryst in view of Rangel (US20190234920).
In regards to Claim 2, Kryst discloses the claimed invention as detailed above in Claim 1. Kryst further teaches “obtaining data that associates at least the location associated with the adjustable mechanical assembly with the one or more values for the one or more second values for the one or more calibration parameters, wherein the determining to update the one or more first values for the one or more calibration parameters to include the one or more second values for the one or more calibration parameters is based at least on the data (The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]).”
Kryst is silent with regards to the language of “obtaining data representing a table”
Rangel teaches “obtaining data representing a table (“calibration can be provided 66 , over the network, to the sensor system 26 —for example by being sent in a network message. The processor 32 of the sensor system 26 may store the calibration into the sensor system's data store 36 —for example in the form of a data table or other stored association.” – [0044]).”
It would have been obvious to one of ordinary skill in the art before the effective skill of the art to modify Kryst to incorporate the teaching of Rangel to utilize a table to store data relating to the calibration of the sensor. By utilizing a table to store data relating to the calibration of a sensor this yields predictable results in the storing and retrieval of calibration data for a sensor.
Claims 4, 10, and 12-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kryst in view of Purdy (US20230058731).
In regards to Claim 4, Kryst in view of Meng discloses the claimed invention as detailed above in Claim 1. Kryst further teaches “determining, using the one or more first sensors of the vehicle, a location associated with the adjustable mechanical assembly located within the interior of the vehicle (The invention relates to a method and a device for monitoring the interior of a vehicle and a vehicle comprising a corresponding device according to the independent claims - [0001]; Advantageously, the method includes an evaluation of measurement data [i.e. first sensor data] from at least one sensor [i.e. one or more first sensors] to detect the position of the rearview mirror [i.e. an adjustable mechanical assembly]. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]; The at least one sensor can be a magnetic angle sensor and/or an accelerometer and/or a force sensor and/or a gyroscope and/or a vibration sensor. A magnetic angle sensor can be arranged, in particular, between the rearview mirror and a foot of the rearview mirror. The aforementioned force sensor can, in particular, be located at one of the bases of the rearview mirror. The gyroscope and/or the vibration sensor and/or the accelerometer may also be located on the foot and/or on the rearview mirror. A magnetic angle sensor and/or an accelerometer have the advantage that both a change in the position of the rearview mirror and the exact deviation from the basic position can be detected - [0011]); and determining, based at least on the second location associated with the adjustable mechanical assembly, the one or more first values for the one or more calibration parameters of the second sensor (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. second sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. second sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]).”
Kryst are silent with regards to the language of “determining, using the one or more first sensors of the vehicle, a second location associated with the assembly located within the interior of the vehicle.”
Purdy teaches “determining, using the one or more first sensors of the vehicle, a second location associated with the assembly located within the interior of the vehicle (“For instance, an example 104 illustrates that a vehicle 106 may determine (e., hypothesize) potential characteristics 108 ( 1 )-( 6 ) (also referred to as “potential characteristics 108 ”) for potential objects located within the environment. In the example of FIG. 1 , the potential characteristics 108 include at least locations and orientation for the potential objects. For example, the potential characteristic 108 ( 1 ) represents a first location and first orientation for a potential object, the potential characteristic 108 ( 2 ) represents a second location and second orientation for a potential object, the potential characteristic 108 ( 3 ) represents a third location and third orientation for a potential object, the potential characteristic 108 ( 4 ) represents a fourth location and a fourth orientation for a potential object, the potential characteristic 108 ( 5 ) represents a fifth location and fifth orientation for a potential object, the potential characteristic 108 ( 6 ) represents a sixth location and a sixth orientation for a potential object. While the example of FIGS. 1 A- 1 B illustrate only six potential characteristics 108 , in other examples, the vehicle 106 may determine any number of potential characteristics 108 for any number of potential objects within the environment” – [0036]).”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kryst to incorporate the teaching of Purdy to utilize multiple locations and orientation of an object with sensors. By utilizing the detection of multiple locations and orientations of a potential object with a sensor, this is an improvement that yields predictable results in the detection of the position and orientation of an objection so that determination of the position and orientation is accurate.
In regards to Claim 10, Kryst teaches “one or more processors (control unit – [0034]) to:
determine, based at least on a first location associated with an adjustable component located within an interior of a machine, one or more first values for one or more calibration parameters of a first sensor attached to the adjustable component (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. first sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. first sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]);
receive sensor data obtained using a second sensor located within the interior of the(The invention relates to a method and a device for monitoring the interior of a vehicle and a vehicle comprising a corresponding device according to the independent claims - [0001]; Advantageously, the method includes an evaluation of measurement data [i.e. sensor data] from at least one sensor [i.e. one or more second sensors] to detect the position of the rearview mirror [i.e. an adjustable mechanical assembly]. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]; The at least one sensor can be a magnetic angle sensor and/or an accelerometer and/or a force sensor and/or a gyroscope and/or a vibration sensor. A magnetic angle sensor can be arranged, in particular, between the rearview mirror and a foot of the rearview mirror. The aforementioned force sensor can, in particular, be located at one of the bases of the rearview mirror. The gyroscope and/or the vibration sensor and/or the accelerometer may also be located on the foot and/or on the rearview mirror. A magnetic angle sensor and/or an accelerometer have the advantage that both a change in the position of the rearview mirror and the exact deviation from the basic position can be detected - [0011]; Furthermore, the method can include an evaluation of measurement data from at least one infrared sensor 14, which is preferably arranged in an A-pillar of the vehicle. - [0032]; Furthermore, the method 100 includes in particular the monitoring of the interior of the vehicle, preferably by means of the measurement data from the camera 11 and the at least one infrared sensor 14 - [0033]);
determine based at least on the sensor data, a location associated with the adjustable component located within the interior of the machine (Advantageously, the method includes an evaluation of measurement data [i.e. sensor data] from at least one sensor [i.e. one or more second sensors] to detect the position of the rearview mirror [i.e. an adjustable mechanical assembly]. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]); and
determine, based at least on the location associated with the adjustable component, one or more second values for the one or more calibration parameters of the first sensor (The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030])”
Kryst is silent with regards to the language of “determine based at least on the sensor data, a second location associated with the component located within the interior of the machine.”
Purdy teaches “determine based at least on the sensor data, a second location associated with the component located within the interior of the machine (“For instance, an example 104 illustrates that a vehicle 106 may determine (e., hypothesize) potential characteristics 108 ( 1 )-( 6 ) (also referred to as “potential characteristics 108 ”) for potential objects located within the environment. In the example of FIG. 1 , the potential characteristics 108 include at least locations and orientation for the potential objects. For example, the potential characteristic 108 ( 1 ) represents a first location and first orientation for a potential object, the potential characteristic 108 ( 2 ) represents a second location and second orientation for a potential object, the potential characteristic 108 ( 3 ) represents a third location and third orientation for a potential object, the potential characteristic 108 ( 4 ) represents a fourth location and a fourth orientation for a potential object, the potential characteristic 108 ( 5 ) represents a fifth location and fifth orientation for a potential object, the potential characteristic 108 ( 6 ) represents a sixth location and a sixth orientation for a potential object. While the example of FIGS. 1 A- 1 B illustrate only six potential characteristics 108 , in other examples, the vehicle 106 may determine any number of potential characteristics 108 for any number of potential objects within the environment” – [0036]).”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kryst to incorporate the teaching of Purdy to utilize multiple locations and orientation of an object with sensors. By utilizing the detection of multiple locations and orientations of a potential object with a sensor, this is an improvement that yields predictable results in the detection of the position and orientation of an objection so that determination of the position and orientation is accurate.
In regards to Claim 12, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “the sensor data includes image data representative of an image (Furthermore, the method 100 includes in particular the monitoring of the interior of the vehicle, preferably by means of the measurement data from the camera 11 and the at least one infrared sensor 14 - [0033];
the one or more processing units are to determine the second location associated with the adjustable component located within the interior of the machine by determining, based at least on the image data, the image depicts the adjustable component at the second location (Furthermore, the device according to the invention comprises an infrared sensor 14. Furthermore, the device 10 includes a control unit 16 for controlling the camera 11, the sensor 13, the infrared sensor 14 and the unit 12 for detecting the position of the rearview mirror (20) - [0034])
In regards to Claim 13, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “receive second sensor data obtained using the first sensor, the second sensor data representative of a sensor representation, wherein the one or more second values for the one or more calibration parameters of the first sensor are further determined based at least on the second sensor data (Calibration can be performed in particular using image recognition. For this purpose, the camera is adjusted to a specific image section before delivery of the vehicle, whereby the image section includes one or more reference points, with the help of which a positional deviation of the rearview mirror can be calculated. Possible reference points could be, for example, the rear wiper and/or the top edge of the sky and/or window cutouts. In particular, appropriate calibration can be carried out at regular intervals and/or directly before an action is to be performed by the vehicle. – [0009]; Furthermore, the alignment can involve a change to a readable area on the camera's sensor. In particular, the scene to be recorded is mapped onto an area of the sensor, whereby the sensor area is larger than the generated image. By changing the area to be read out, the area to be recorded in the interior can be adjusted and the camera can be aligned accordingly. – [0017]).”
In regards to Claim 14, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “wherein the one or more processors are further to: determine that a portion of the sensor representation depicts a feature located within the interior of the machine, wherein the one or more second values for the one or more calibration parameters of the first sensor are determined based at least on the second location associated with the adjustable component and the portion of the sensor representation (The invention relates to a method and a device for monitoring the interior of a vehicle and a vehicle comprising a corresponding device according to the independent claims - [0001]; Advantageously, the method includes an evaluation of measurement data from at least one sensor to detect the position of the rearview mirror. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]; The at least one sensor can be a magnetic angle sensor and/or an accelerometer and/or a force sensor and/or a gyroscope and/or a vibration sensor. A magnetic angle sensor can be arranged, in particular, between the rearview mirror and a foot of the rearview mirror. The aforementioned force sensor can, in particular, be located at one of the bases of the rearview mirror. The gyroscope and/or the vibration sensor and/or the accelerometer may also be located on the foot and/or on the rearview mirror. A magnetic angle sensor and/or an accelerometer have the advantage that both a change in the position of the rearview mirror and the exact deviation from the basic position can be detected - [0011]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]).”
In regards to Claim 15, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “wherein the one or more second values for the one or more calibration parameters of the first sensor are further determined based at least on the one or more first values for the one or more calibration parameters of the first sensor (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. second sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. second sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]).”
In regards to Claim 16, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “wherein the one or more processors are further to: determine, based at least on second sensor data obtained using the first sensor and the one or more second values for the one or more calibration parameters, a location of an object within the interior of the machine (Advantageously, the method includes an evaluation of measurement data from at least one sensor to detect the position of the rearview mirror. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]; The at least one sensor can be a magnetic angle sensor and/or an accelerometer and/or a force sensor and/or a gyroscope and/or a vibration sensor. A magnetic angle sensor can be arranged, in particular, between the rearview mirror and a foot of the rearview mirror. The aforementioned force sensor can, in particular, be located at one of the bases of the rearview mirror. The gyroscope and/or the vibration sensor and/or the accelerometer may also be located on the foot and/or on the rearview mirror. A magnetic angle sensor and/or an accelerometer have the advantage that both a change in the position of the rearview mirror and the exact deviation from the basic position can be detected - [0011]); and cause, based at least on the location of the object, performance of one or more operations associated with the machine (Based on the calibration of the mirror position, an error-free assignment of the driver for driver observation is possible [i.e. performance of one or more operations associated with the vehicle]. It is also possible to create an optimal image for video telephony, so that the recorded images can be optically aligned in such a way that they are independent of the position of the rearview mirror. Furthermore, a relationship to the instrument panel can be determined for distance detection between the occupants and an airbag, and individual control of the restraint systems can be achieved. – [0012]).”
In regards to Claims 17 and 20, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst is silent with regards to the language of “wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations; a system implemented using an edge device;
a system implemented using a robot; a system implementing one or more large language models (LLMs); a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources .”
Purdy further teaches “wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations; a system implemented using an edge device;
a system implemented using a robot; a system implementing one or more large language models (LLMs); a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (autonomous vehicle – [0001]; semi-autonomous vehicle – [0063]).”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kryst in view of Purdy to incorporate the further teaching of Purdy to integration into a semi-autonomous vehicle. By utilizing this system in an semi-autonomous vehicle, this is an improvement that yields predictable results in the safety of people in the vehicle, as Kryst details system for control of safety systems in [0005] and Purdy details safety of the vehicle in [0031].
In regards to Claim 18, Kryst teaches “One or more processors comprising processing circuitry (control unit – [0034]) to:
determine, based at least on first sensor data obtained using a first sensor of a machine, a first location associated with an adjustable component located within an interior of the machine (The invention relates to a method and a device for monitoring the interior of a vehicle and a vehicle comprising a corresponding device according to the independent claims - [0001]; Advantageously, the method includes an evaluation of measurement data [i.e. first sensor data] from at least one sensor [i.e. one or more first sensors] to detect the position of the rearview mirror [i.e. an adjustable mechanical assembly]. The sensor can be used exclusively to detect the position of the rearview mirror. In particular, it concerns exactly one or more sensors. These are designed to detect mirror movement and thus a change in the position of the rearview mirror. Such a sensor has the advantage that calibration does not need to take place before every action or at regular intervals, but that it is sufficient if calibration is only carried out once after a change in the position of the rearview mirror. - [0010]; The at least one sensor can be a magnetic angle sensor and/or an accelerometer and/or a force sensor and/or a gyroscope and/or a vibration sensor. A magnetic angle sensor can be arranged, in particular, between the rearview mirror and a foot of the rearview mirror. The aforementioned force sensor can, in particular, be located at one of the bases of the rearview mirror. The gyroscope and/or the vibration sensor and/or the accelerometer may also be located on the foot and/or on the rearview mirror. A magnetic angle sensor and/or an accelerometer have the advantage that both a change in the position of the rearview mirror and the exact deviation from the basic position can be detected - [0011]);
determine, based at least on the first location associated with the adjustable component, one or more first values for one or more parameters that calibrate a second sensor attached to the adjustable component with respect to a coordinate system of the machine (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. second sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. second sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]); and
determine, based at least on the location associated with the adjustable component, to update the one or more first values to include one or more second values for the one or more parameters that calibrate the second sensor with respect to the coordinate system of the machine (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. second sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. second sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030])”
Kryst is silent with regards to the language of “determine, based at least on second sensor data obtained using the first sensor, a second location associated with the adjustable component.”
Purdy teaches “determine, based at least on second sensor data obtained using the first sensor, a second location associated with the adjustable component (“For instance, an example 104 illustrates that a vehicle 106 may determine (e., hypothesize) potential characteristics 108 ( 1 )-( 6 ) (also referred to as “potential characteristics 108 ”) for potential objects located within the environment. In the example of FIG. 1 , the potential characteristics 108 include at least locations and orientation for the potential objects. For example, the potential characteristic 108 ( 1 ) represents a first location and first orientation for a potential object, the potential characteristic 108 ( 2 ) represents a second location and second orientation for a potential object, the potential characteristic 108 ( 3 ) represents a third location and third orientation for a potential object, the potential characteristic 108 ( 4 ) represents a fourth location and a fourth orientation for a potential object, the potential characteristic 108 ( 5 ) represents a fifth location and fifth orientation for a potential object, the potential characteristic 108 ( 6 ) represents a sixth location and a sixth orientation for a potential object. While the example of FIGS. 1 A- 1 B illustrate only six potential characteristics 108 , in other examples, the vehicle 106 may determine any number of potential characteristics 108 for any number of potential objects within the environment” – [0036]).”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kryst to incorporate the teaching of Purdy to utilize multiple locations and orientation of an object with sensors. By utilizing the detection of multiple locations and orientations of a potential object with a sensor, this is an improvement that yields predictable results in the detection of the position and orientation of an objection so that determination of the position and orientation is accurate.
In regards to Claim 19, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “determine that the adjustable component moved from the first location within the interior of the machine to the second location within the interior of the machine, wherein the determination to update the one or more first values to include the one or more second values occurs in response to the adjustable component being moved from the first location to the second location (The aforementioned problem is solved by a method for monitoring the interior of a vehicle, which includes taking pictures using a camera, wherein the camera [i.e. second sensor] is arranged behind a mirror glass of a movable rearview mirror [i.e. second sensor attached to the adjustable mechanical assembly] of the vehicle. – [0004]; The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]).”
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kryst in view of Purdy as applied to claim 10 above, and further in view of Rangel (US20190234920).
In regards to Claim 11, Kryst in view of Purdy discloses the claimed invention as detailed above. Kryst further teaches “wherein the one or more processors are further to: obtain data that associates at least the second location associated with the adjustable component with the one or more second values for the one or more calibration parameters, wherein the one or more second values for the one or more calibration parameters of the first sensor are further determined based at least on the data (The procedure may include, in particular, aligning the camera, especially within the mirror housing. This serves in particular to counteract changes in the position of the rearview mirror, so that after alignment the camera again takes pictures that the camera would take if the rearview mirror had not changed its position and was preferably in its basic position. In other words, the camera's alignment serves to adjust the camera's position in response to a change in the position of the rearview mirror. In particular, the camera is aligned as soon as a change in the position of the movable rearview mirror is detected [i.e. updating the calibration parameters; as the camera position is adjusted, the coordinate system positions of x-position and y-position are adjusted, where the x-position would correspond to the first calibration parameter and the y position would correspond to the second calibration parameter] - [0013]; The alignment of the camera can, in particular, include a change in the position of the camera within the mirror housing. In other words, it is a movement, in particular a pivoting, of the camera in relation to the mirror housing and thus to the looking mirror. – [0014]; Furthermore, the alignment can include a change in the position of a sensor within the camera. - [0016]; Additionally or alternatively, the calibration can include evaluating 104 measurement data from at least one sensor 13 to detect the position of the rearview mirror (20) - [0030]).”
Kryst in view of Purdy is silent with regards to the language of “obtaining data representing a table”
Rangel teaches “obtaining data representing a table (“calibration can be provided 66 , over the network, to the sensor system 26 —for example by being sent in a network message. The processor 32 of the sensor system 26 may store the calibration into the sensor system's data store 36 —for example in the form of a data table or other stored association.” – [0044]).”
It would have been obvious to one of ordinary skill in the art before the effective skill of the art to modify Kryst in view of Purdy to incorporate the teaching of Rangel to utilize a table to store data relating to the calibration of the sensor. By utilizing a table to store data relating to the calibration of a sensor this yields predictable results in the storing and retrieval of calibration data for a sensor.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Kryst in view of Heidi (US20160343136).
In regards to Claim 9, Kryst discloses the claimed invention as detailed above. Kryst are silent with regards to the language of “wherein the one or more second values for the one or more calibration parameters comprise at least one of: one or more values associated with one or more translation parameters for calibrating the second sensor with respect to a coordinate system associated with the vehicle; or one or more values associated with one or more rotational parameters for calibrating the second sensor with respect to the coordinate system associated with the vehicle.”
Heidi teaches “wherein the one or more second values for the one or more calibration parameters comprise at least one of: one or more values associated with one or more translation parameters for calibrating the second sensor with respect to a coordinate system associated with the vehicle; or one or more values associated with one or more rotational parameters for calibrating the second sensor with respect to the coordinate system associated with the vehicle (extrinsic camera parameters include the rotational matrix Rij and the translation vector Tij, which relates the real world 3D coordinate system and the camera 3D system – [0046]).”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kryst to incorporate the teaching of Heidi to utilize a rotational matrix and translation vector for the calibration of a camera between different coordinate systems. By utilizing rotational matrix and translation vector this is an improvement that yields predictable results in the conversion between coordinate systems.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOSSEF KORANG-BEHESHTI whose telephone number is (571)272-3291. The examiner can normally be reached Monday - Friday 10:00 am - 6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Catherine Rastovski can be reached at (571) 270-0349. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOSSEF KORANG-BEHESHTI/Primary Examiner, Art Unit 2857
Prosecution Insights