DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-22 are pending with claims 1, 9-11, 15-17, and 21 being amended.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Peterson et al. (US 20160243988 A1) in view of Kim et al. (US 20180052457 A1).
Concerning claims 1, 11, and 15, Peterson et al. (hereinafter Peterson) teaches one or more processors, comprising circuitry to:
receive one or more sensory inputs from one or more visual sensors attached to a device via an extensible apparatus (figs. 1-6: camera module 20, ¶0028: camera module 20 (such as a sideward and/or rearward facing imaging sensor or camera) may provide an image output for a vehicle vision system, such as a lane departure warning system or object detection system or blind zone alert system or surround view vision system other vehicle vision system or the like, and may utilize aspects of various imaging sensors or imaging array sensors or cameras or the like; ¶0047: “The camera module may be extended and pivoted via any suitable extension/pivoting means, such as an extendable retractable arm and a pivoting motor or the like”);
identify, from the one or more sensory inputs, one or more objects obstructing a view of the device from a current position of at least one visual sensor of the one or more visual sensors (¶0074: The image sensor provides image output as part of an object detection system or blind zone alert system that detect obstacles and/or objects around the environment of the vehicle); and
cause the extensible apparatus upon which the one or more visual sensors are affixed to extend in a linear direction away from a side of the device from the current position to a different position relative to the device to obtain a view around the identified one or more objects obstructing the view of the device (¶0035 & ¶0065: The camera is automatically extended, retracted and/or pivoted in response to a triggering event in order to see further around an object or obstacle at or near the vehicle. Figs. 1 & 2 show camera module 20 attached to an extensible apparatus in order to extend in a linear direction away from a side of the device.). Not explicitly taught is us[ing] one or more machine learning models to generate information indicating an adjustment to the current position of the at least one visual sensor to optimize visual sensing of the one or more visual sensors based, at least in part, on the received one or more sensory inputs, the identified one or more objects, and environmental conditions and controlling the extensible apparatus upon which the one or more visual sensors are affixed based, at least in part, on the information generated by the one or more machine learning models.
Kim et al. (hereinafter Kim), in the same field of endeavor, teaches the use of one or more machine learning models to generate information indicating an adjustment to the current position of the at least one visual sensor to optimize visual sensing of the one or more visual sensors based, at least in part, on the received one or more sensory inputs, the identified one or more objects, and environmental conditions (fig. 10: operation 1020, ¶0084: a driving situation model or other machine or deep learning may be used in estimating the driving situation based on information from sensors, determined conditions and previous training data; ¶0083: inputs from various sensors (e.g., speed, GPS, etc.), identified objects (e.g., obstacles or obtruding vehicles) and environmental conditions (e.g., weather, road conditions, etc.) may be used in estimating the driving situation. Based on the driving situation, controller 120 determines or sets the appropriate parameter (i.e., optimized visual sensing) associated with the width of the stereo camera), and controlling the apparatus upon which the one or more visual sensors are affixed to obtain another view that is different from the current view of the one or more visual sensors based, at least in part, on the information generated by the one or more machine learning models (¶0086: “…the controller 120 may predict a driving direction of the vehicle using information from the steering device or the direction indicating device and adjust a parameter for controlling a rotational angle and/or a horizontal visual field width of the stereo camera based on the predicted driving direction of the vehicle.”. That is to say, the controller adjusts a parameter other than the stereo width (e.g., a rotational angle) of at least one visual sensor based on the predicted driving direction. The predicted driving direction is determined using a driving situation model or other deep learning, machine learning, or hierarchical analytics that may be trained or designed based on such training data (see, ¶0084).). It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the teachings of Kim into the Peterson invention in order to estimate the driving situation of the vehicle and control the movement of the extensible apparatus (i.e., “cause the extensible apparatus upon which the one or more visual sensors are affixed to extend in a linear direction away from a side of the device from the current position to a different position relative to the device to obtain a view around the identified one or more objects obstructing the view of the device”) based on the estimating driving situation that is determined by one or more machine learning models. The addition of machine learning models would allow the Peterson invention to be trained and/or designed based on training data in order to make decisions and/or predictions without explicit instructions.
Concerning claims 2 and 12, Kim teaches the one or more processors of claim 1, further comprising using the one or more processors to determine environmental conditions based at least in part on receiving one or more visual sensor inputs from the one or more visual sensors (¶0083: Also, the controller 120 may predict the road condition based on a result obtained by analyzing an image capturing a current driving road and/or the weather information.).
Concerning claim 3, Peterson further teaches the one or more processors of claim 2, wherein the one or more visual sensors include one or more cameras (figs. 1-6: camera module 20 & ¶0028).
Concerning claim 4, Peterson further teaches the one or more processors of claim 2, wherein the one or more visual sensor inputs include at least one of: one or more images of an environment surrounding the one or more visual sensors (figs. 13A-13B; ¶0035) or one or more video frames of an environment surrounding the one or more visual sensors (figs. 13A-13B; ¶0035).
Concerning claim 5, Kim further teaches the one or more processors of claim 2, wherein using the processor to determine the environmental conditions includes:
inputting the one or more visual sensor inputs to the one or more machine learning models (¶¶0083-0084), and
receiving as output from the one or more machine learning models the environmental conditions (¶¶0083-0084).
Concerning claim 6, Kim further teaches the one or more processors of claim 5, wherein the one or more machine learning models are trained using training data to learn the environmental conditions from visual sensor inputs (¶0084: previous training data and being designed based on such training data (e.g., information from such differing sensors or determined conditions and based on previous training data)).
Concerning claim 7, Kim further teaches the one or more processors of claim 2, wherein the environmental conditions include a state of an environment surrounding the one or more visual sensors (¶0083: road conditions, weather, etc.).
Concerning claim 8, Peterson further teaches the one or more processors of claim 1, wherein adjusting the one or more visual sensors comprises using a position control signal to instruct the one or more visual sensors from a current position to an extended position of a defined amount (¶0046).
Concerning claim 9, Peterson further teaches the one or more processors of claim 1, wherein causing the at least one visual sensor to be adjusted includes causing the at least one visual sensor to change position rotationally (¶0038).
Concerning claim 10, Peterson further teaches the one or more processors of claim 1, wherein causing the at least one visual sensor to be adjusted includes causing the at least one visual sensor to change position linearly (figs. 1-2: retracted and extended positions of camera module 20).
Concerning claim 13, Kim further teaches the non-transitory computer readable medium of claim 11, wherein visual information of the environmental conditions is displayed to a user (¶0059).
Concerning claim 14, Kim further teaches the non-transitory computer readable medium of claim 13, wherein the visual information is output to an automated system that makes a decision based at least in part on the visual information (¶0087).
Concerning claim 16, Peterson further teaches the method of claim 15, wherein causing the extensible apparatus to extend includes changing at least one visual sensor from the current position to an optimal position and includes causing the at least one visual sensor to change linearly with respect to the current position (figs. 1-2: retracted and extended positions of camera module 20; ¶0046).
Concerning claim 17, Peterson further teaches the method of claim 16, wherein causing the at least one visual sensor to change from the current position to the optimal position includes causing the at least one visual sensor to change rotationally with respect to the current position (¶0038; ¶¶0046-0047).
Concerning claim 18, Kim teaches the method of claim 15, further comprising using the one or more machine learning models, by a vehicle, to determine visual information of environmental conditions surrounding the vehicle (¶0084: previous training data and being designed based on such training data (e.g., information from such differing sensors or determined conditions and based on previous training data)).
Concerning claim 19, Peterson, now incorporating the teachings of Kim, further teaches the method of claim 16, further comprising: using one or more additional machine learning models to adjust the one or more visual sensors from the optimal position to a new optimal position based at least in part on a state of an environment surrounding the one or more visual sensors (Peterson, ¶0065: the extension, retraction and/or pivoting of the camera are reactive to detections at or around the vehicle and are thus updated accordingly (e.g., detecting the blind spot region (see, ¶0005)); Kim, fig. 10: operation 1020, ¶0084: a driving situation model or other machine or deep learning may be used in estimating the driving situation based on information from sensors, determined conditions and previous training data).
Concerning claim 20, Peterson further teaches the method of claim 19, wherein the optimal position is an extended position and the new optimal position is a retracted position (¶0005: a retracted position may be suitable for blind spot detection).
Claims 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Peterson et al. (US 20160243988 A1) in view of Kim et al. (US 20180052457 A1) and Atwater et al. (US 20190340440 A1).
Concerning claim 21, Peterson in view of Kim teaches the system of claim 1. Not explicitly taught is the system, wherein output of the one or more machine learning models indicates how to adjust the at least one visual sensor.
Atwater et al. (hereinafter Atwater), in a similar field of endeavor, teaches using the output of one or more machine learning models that indicate how to adjust the one or more visual sensors (¶¶0074-0075: Machine learning techniques are used to learn preferred camera positions and locations based on the training data. The monitoring system may also learn how to position cameras at certain locations and the camera settings that may be utilized to obtain quality images.). It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to add the features of Atwater to the Peterson in view of Kim invention in order to adjust the visual sensors to obtain high quality images (Atwater, ¶0075).
Concerning claim 22, Peterson in view of Kim teaches the method of claim 15. Not explicitly taught is the system, wherein the one or more machine learning models are used to infer the adjustment.
Atwater et al. (hereinafter Atwater), in a similar field of endeavor, teaches using the one or more machine learning models infer the adjustment (¶¶0074-0075: Machine learning techniques are used to learn preferred camera positions and locations based on the training data. The monitoring system may also learn how to position cameras at certain locations and the camera settings that may be utilized to obtain quality images.). It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to add the features of Atwater to the Peterson in view of Kim invention in order to adjust the visual sensors to obtain high quality images (Atwater, ¶0075).
Response to Arguments
Applicant’s arguments, see pages 8-10 of the remarks, filed 07/24/2026, with respect to the rejection of claims 1-22 under 35 U.S.C. §103 have been fully considered, but they are not persuasive. The rejections of the claims in view of Peterson and Kim are maintained for the reasons provided below.
In response to the Kim prior art, the Applicant alleges
“Accordingly, Kim's machine learning model is used to estimate a driving condition or driving situation. Kim does not disclose or suggest using one or more machine learning models to "generate information indicating an adjustment to the current position of the at least one visual sensor to optimize visual sensing of the one or more visual sensors," as recited in claim 1.”
Although paragraph [0106] of Kim discloses that "the controller may calculate reaction distances ... and determine the appropriate parameter associated with the width of the stereo camera based on the appropriate reaction distance," Kim does not disclose that such parameters are generated using a machine learning model.
Moreover, even assuming arguendo that Kim uses machine learning in connection with determining such parameters, the resulting adjustment concerns the width of a stereo camera arrangement. That disclosure is materially different from the claimed elements that recite "caus[ing] the extensible apparatus upon which the one or more visual sensors are affixed to extend in a linear direction away from a side of the device from the current position to a different position relative to the device." Changing stereo camera width is not the same as extending an extensible apparatus in a linear direction away from a side of the device, as recited in claim 1.
This is not persuasive. Kim, in at least paragraph [0086], teaches controlling another parameter (e.g., rotational angle) of the stereo camera based on the driving situation as well as the parameter associated with the width of the stereo camera. The driving situation is determined based on information provided to a driving situation model or other deep learning, machine learning, or hierarchical analytics that may be trained or designed based on such training data. Based on this teaching, one of ordinary skill in the art would understand that the at least one or more machine learning models are used to “generate information indicating an adjustment to the current position of the at least one visual sensor to optimize visual sensing of the one or more visual sensors”. Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Peterson and Kim by using one or more machine models to estimate a current driving situation and adjust the current view of the at least one visual sensor by manipulating an apparatus. The addition of machine learning models would allow the Peterson invention to be trained and/or designed based on training data in order to make decisions and/or predictions without explicit instructions.
The Applicant further alleges
“Applicant further submits that the proposed combination of Peterson and Kim is improper due to a lack of motivation to combine. Peterson is directed to extending/retracting a camera assembly based on user inputs or predefined triggering events, whereas Kim is directed to estimating driving situations using machine learning models. The Office has not identified any teaching or suggestion in Kim that would modify Peterson such that Peterson's extensible apparatus would operate "based, at least in part, on the information generated by the one or more machine learning models," as recited in claim 1.”
This argument has been considered but it is not persuasive. The benefits of using machine learning models in improved decision making and automating tasks have been well documented and are well-known in the art. The addition of machine learning models would allow the Peterson invention to be trained and/or designed based on training data in order to make decisions and/or predictions without explicit instructions. Furthermore, the Applicant has not provided a special definition for a “machine learning model” that clearly redefines the plain meaning; therefore, it is given its ordinary and customary meaning consistent with the interpretation that those skilled in the art would reach. Accordingly, the use of one or more machine models to perform the automatic extension/retraction/pivoting of the at least one or more visual sensor is well within the capabilities of one skilled in the art.
Regarding the combination of Peterson and Kim, the applicant alleges
“Further, even if Kim were combined with Peterson, the combination would be incompatible as Kim's alleged machine learning outputs relate to driving-situation estimation or stereo-camera-width adjustment, not to generating information used to extend an extensible apparatus "in a linear direction away from a side of the device" to "obtain a view around the identified one or more objects obstructing the view of the device," as claimed.”
This is not persuasive. In response to Applicant’s assertion that Peterson and Kim are incompatible, it has been held that a prior art reference must either be in the field of the inventor’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the inventor was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). In this case, Peterson and Kim are both directed to operating at least one visual sensor on a vehicle in an attempt to improve the field of view of said at least one visual sensor based on various situations. As mentioned above, Kim, in at least paragraph [0086], teaches controlling another parameter (e.g., rotational angle) of the stereo camera based on the driving situation as well as the parameter associated with the width of the stereo camera. The driving situation is determined based on information provided to a driving situation model or other deep learning, machine learning, or hierarchical analytics that may be trained or designed based on such training data. Based on this teaching, one of ordinary skill in the art would understand that the at least one or more machine learning models are used to “generate information indicating an adjustment to the current position of the at least one visual sensor to optimize visual sensing of the one or more visual sensors”. Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Peterson and Kim by using one or more machine models to control the extension/retraction mechanism in a linear direction away from a side of the device.
Applicant argues that independent claims 11 and 15 as well as all claims that depend from one of claims 1, 11 or 15 are allowable for the same reasons provided in response to the rejection of independent claim 1. The rejection of these claims is maintained for the same reasons presented above and will not be repeated for the sake of brevity.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES M ANDERSON II whose telephone number is (571)270-1444. The examiner can normally be reached Monday - Friday 10AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BRIAN PENDLETON can be reached at 571-272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/James M Anderson II/Primary Examiner, Art Unit 2425
