DETAILED ACTION
Claims 1-24 are pending in the application.
Response to Arguments
Applicant’s arguments with respect to claims pending have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-4, 6-9, 11-14, 16-18, and 20-23 are rejected under 35 U.S.C. 103 as being unpatentable over Cai et al (US 2021/0337117 A1) in view of Yamashita (US 2022/0353984 A1).
As per claim 1, Cai discloses a vehicle (figs. 1 and 9, image capture and processing system 100, computing system 900, para 0105, …an autonomous vehicle), comprising:
at least one computer-readable medium storing computer-executable instructions (fig. 1, image capture and processing system 100, RAM 140);
at least one processor communicatively coupled to at least one camera and configured to execute the computer executable instructions (fig. 1, image capture and processing system 100, image processor 150 is connected to RAM 140 and image sensor 130), the execution carrying out operations including:
obtaining raw data associated with a raw image from at least one camera (fig. 1, image sensor 130 may capture raw image data, para 0035);
generating an updated image based on the raw image from the at least one camera, wherein the updated image represents photons received by the at least one camera (fig. 1, image sensor 130 captures raw image data and is updated by the image signal processor (ISP) 154 based on light/photons from image sensor 130);
providing the updated image to an image and signal processing pipeline (fig. 1, ISP 154, updated image goes through processing blocks, see para 0035 and 0053).
Cai fails to teach processing the updated image using the image signal processing pipeline adapted according to control parameters determined by front-end processing logic, wherein a planned use of the updated image controls steps of processing logic performed by the image signal processing pipeline and transmitting instructions for causing a vehicle to operate based at least in part on information obtained from an analysis of the processed image.
However, Yamashita discloses a drive control system 100, using raw image data captured by imaging module 2 having a processing unit 110 (i.e. image signal processing pipeline) for processing the image signal including a mapping processing unit 114 and path planning unit 115 for determining the usage of the image data captured, and transmitting said processed image signal data to automobile M for controlling features in the automobile such as a driving-force generating mechanism M11, braking mechanism M12, and steering mechanism M13 and other control parameters of automobile M (Yamashita, figs. 12-14, drive control system 100, processing unit 110, automobile M, para [0153-0155]).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Cai in view of Yamashita as a whole, by incorporating the drive control system/processing unit as taught by Yamashita into the computing system for autonomous vehicles as taught by Cai, because doing so would provide a more efficient way of controlling the autonomous vehicle, thus enhancing the driving capabilities of the autonomous vehicle.
As per claim 2, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 1, the operations comprising:
identifying the steps of the image signal processing pipeline to be omitted in processing of the updated image (Cai, fig. 1, image capture and processing system 100, ISP 154, the ability to select which signal processing to perform, para 0053).
As per claim 3, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 2, wherein the processing of the updated image omits the steps identified for omission (Cai, fig. 1, image capture and processing system 100, ISP 154, the ability to select which signal processing to perform, and which signal processing not to perform, para 0053).
As per claim 4, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 1, wherein identifying steps of the image signal processing pipeline to be omitted in processing of the updated image comprises one or more hardware components to be omitted (Cai, fig. 1, image capture and processing system 100, ISP 154, should be noted ISP 154 can include multiple modules, i.e. hardware, para 0053).
As per claim 6, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 1, wherein the image signal processing pipeline comprises one or more hardware components (Cai, fig. 1, image capture and processing system 100, ISP 154, should be noted ISP 154 can include multiple modules, i.e. hardware, para 0053, also see Yamashita, fig. 14, processing unit 110).
As per claim 7, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 1, wherein the image signal processing pipeline comprises one or more SoCs (Cai, fig. 1, image capture and processing system 100, image processor 150, para 0046).
As per claim 8, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 1, wherein the at least one camera approximates an amount of photons captured to generate the updated image (Cai, fig. 1, image capture device 105A, para 0050 and 0051).
As per claim 9, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the vehicle of claim 1, wherein the raw image data is obtained by controlling an acquisition time of the at least one camera (Cai, fig. 1, image capture device 105A, control mechanisms 120, exposure 125A).
As per claim 11, the combined teachings of Cai in view of Yamashita, as a whole, further discloses a method comprising:
obtaining raw data associated with a raw image from at least one camera;
generating an updated image based on the raw image from the at least one camera, wherein the updated image represents photons received by the at least one camera;
providing the updated image to an image signal processing pipeline;
processing the updated image using the image signal processing pipeline adapted according to control parameters determined by front-end processing logic, wherein a planned use of the updated image controls steps of processing logic performed by the image signal processing pipeline, and
transmitting instructions for causing a vehicle to operate based at least in part on information obtained from an analysis of the processed image (claim limitations have been discussed and rejected, see claim 1 above).
As per claim 12, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the method of claim 11, comprising:
identifying steps of the image signal processing pipeline to be omitted in processing of the updated image (claim limitations have been discussed and rejected, see claim 2 above).
As per claim 13, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the method of claim 12, wherein processing of the updated image omits the steps identified for omission (claim limitations have been discussed and rejected, see claim 3 above).
As per claim 14, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the method of claim 12, wherein identifying steps of the image signal processing pipeline to be omitted in processing of the updated image comprises one or more hardware components to be omitted (claim limitations have been discussed and rejected, see claim 4 above).
As per claim 16, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the method of claim 11, wherein the image signal processing pipeline comprises one or more hardware components (claim limitations have been discussed and rejected, see claim 6 above).
As per claim 17, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the method of claim 11, wherein the image signal processing pipeline comprises one or more SoCs (claim limitations have been discussed and rejected, see claim 7 above).
As per claim 18, the combined teachings of Cai in view of Yamashita, as a whole, further discloses the method of claim 11, wherein the at least one camera approximates an amount of photons captured to generate the updated image (Cai, fig. 1, image capture device 105A, para 0050 and 0051).
As per claim 20, the combined teachings of Cai in view of Yamashita, as a whole, further discloses a non-transitory computer-readable storage medium comprising at least one program for execution by one or more processors of a first device, the at least one program including instructions which, when executed by the one or more processors (Cai, para 0108), cause the first device to perform operations comprising:
obtaining raw data associated with a raw image from at least one camera;
generating an updated image based on the raw image from the at least one camera, wherein the updated image represents photons received by the at least one camera;
providing the updated image to an image and signal processing pipeline;
processing the updated image using image signal processing pipeline adapted according to control parameters determined by front-end processing logic wherein a planned use of the updated image controls steps of processing logic performed by the image signal processing pipeline; and
transmitting instructions for causing a vehicle to operate based at least in part on information obtained from an analysis of the processed image (claim limitations have been discussed and rejected, see claim 1 above).
As per claim 21, the vehicle of claim 1, wherein the planned use indicates the updated image is displayed on a user interface for human visualization (Cai, fig. 1, I/O 160, para 0048 and 105).
As per claim 22, the vehicle of claim 1, wherein the planned use indicates the updated image further processed by a perception or planning system (Yamashita, fig. 14, drive control system 100, processing unit 110, path planning unit 115).
As per claim 23, the vehicle of claim 1, wherein the image signal processing pipeline comprises at least one System-on-Chip (SoC), and wherein the front-end processing logic is implemented on the at least one SoC (Cai, para 0046).
4. Claims 5, 15, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Cai et al (US 2021/0337117 A1) and Yamashita (US 2022/0353984 A1), in further view of Gray (US 10,007,269 B1).
As per claim 5, the vehicle of claim 2, wherein steps of the image signal processing pipeline to be omitted in processing of the updated image are determined based on a state of the vehicle.
Cai in view of Yamashita, as a whole, fails to teach the limitations recited above in claim 5. However, Gray discloses an collision-avoidance system wherein based on image frames captured from the camera(s) and the image analysis of the image frames, the state of the vehicle can be determined (Gray, fig. 1, autonomous-capable vehicle 10, collision-avoidance system 100, col. 8 lines 37-col. 9 lines 41).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Cai and Yamashita, in further view of Gray, as a whole, by incorporating the collision-avoidance system as taught by Gray, into the computing system for autonomous vehicles as taught by Cai and Yamashita, because doing so would provide a more efficient way to determine a state of the vehicle through the processing of images captured.
As per claim 15, the combined teachings of Cai and Yamashita, in further view of Gray, as a whole, further disclose the method of claim 12, wherein identifying steps of the image signal processing pipeline to be omitted in processing of the updated image are determined based on a state of the vehicle (claim limitations have been discussed and rejected, see claim 5 above).
As per claim 24, the combined teachings of Cai and Yamashita, in further view of Gray, as a whole, further disclose the vehicle of claim 1, wherein adapting the image signal processing pipeline comprises omitting at least one processing step of the image signal processing pipeline when associated processing information indicates the updated image is processed for vehicle navigation (claim limitations have been discussed and rejected, see claim 5 above, as well as Cai, para 0113).
5. Claims 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Cai et al (US 2021/0337117 A1) and Yamashita (US 2022/0353984 A1), in further view of Schofield et al (US 8,599,001 B2).
As per claim 10, the vehicle of claim 1, wherein the raw image data comprises an approximation of an amount of photons.
Cai in view of Yamashita as a whole, fails to teach the limitations as recited above in claim 10. However, Schofield discloses a vehicular vision system comprising a photosensor array 32 having the ability to approximate light (i.e. photons) (Schofield, fig. 5, photosensor array 32, col. 13 lines 30-35).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Cai and Yamashita, in further view of Schofield, as a whole, by incorporating the photosensor array as taught by Schofield, into the computing system for autonomous vehicles as taught by Cai and Yamashita, because doing so would provide a more efficient way of imaging a scene.
As per claim 19, Cai in view of Yamashita, as a whole, further discloses the method of claim 11, wherein the raw image data is obtained by controlling an acquisition time of the at least one camera (Cai, fig. 1, image capture device 105A, control mechanisms 120, exposure 125A).
Cai in view of Yamashita, as a whole, fails to teach comprises an approximation of an amount of photons. However, Schofield discloses a vehicular vision system comprising a photosensor array 32 having the ability to approximate light (i.e. photons) (Schofield, fig. 5, photosensor array 32, col. 13 lines 30-35).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Cai and Yamashita, in further view of Schofield, as a whole, by incorporating the photosensor array as taught by Schofield, into the computing system for autonomous vehicles as taught by Cai and Yamashita, because doing so would provide a more efficient way of imaging a scene.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN H MOREHEAD III whose telephone number is (571)270-3845. The examiner can normally be reached M - F 0930-1800 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN H MOREHEAD III/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights