DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 of US Application No. 19/108,943, filed on 05 March 2025, are currently pending and have been examined. Applicant amended claims 1, 3, 4, 6, 8, 9, 11, 12, 19, and 20 via preliminary amendment.
Information Disclosure Statement
The Information Disclosure Statement filed on 05 March 2025 has been considered. An initialed copy of form 1449 is enclosed herewith.
Claim Objections
Claims 2-18 are objected to because of the following informalities:
Claims 2-17 recite “A control system of claim . . .”. Claims 2-17 depend from claim 1 and should recite – The control system of claim –.
Claim 18 recites “a plurality of imaging sensors”. However, claim 18 depends from at least claim 1, which already recites “a plurality of imaging sensors” in line 4. Claim 18 should recite – the plurality of imaging sensors –.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-7, 11, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kriel et al. (US 2014/0203959 A1, “Kriel”).
Regarding claims 1 and 20, Kriel discloses an object recognition system having radar and camera input and teaches:
one or more controllers, and being configured to:
receive image data from each of a plurality of imaging sensors associated with the agricultural machine, the imaging sensors being configured to image respective imaging regions which at least partly overlap (radar devices 120a-120h – see at least Fig. 1 and ¶ [0015]; cameras 140a-140d – see at least Fig. 1 and ¶ [0015]; radar interface 205 and camera interface 206 may receive analog signals from their respective devices and convert them to digital signals which may be processed by the other modules of the object recognition system 200);
analyze the image data from each sensor to determine, for each sensor, an identity for a common object within the respective imaging regions of the imaging sensors (radar devise 120 detect objects in their respective fields of view – see at least ¶ [0016]; cameras 140 may detect objects in their respective fields of view – see at least ¶ [0016]; process image data to detect objects – see at least ¶ [0017]; object recognition system 200, via object detector 215, processes radar and image data to detect objects – see at least Fig. 2 and ¶ [0024]; discriminator 220 classifies the detected object – see at least Fig. 2 and ¶ [0028]);
evaluate the determined identities from each of the sensors to determine a configuration for a user interface of the operator assistance system (alert processor 250 may analyze the object and determine if an alert needs to be generated – see at least ¶ [0040]; display a first alert or second alert based on type of object – see at least ¶ [0040]; display 260 may provide indication boxes 430, 440 based on the type of object); and
generate and output one or more control signals for controlling the user interface in accordance with the determined configuration (processor executes instructions to render graphics and images on display 260, e.g., image 420, in response to the alert – see at least ¶ [0042]).
Regarding claim 2, Kriel further teaches:
wherein each of the imaging sensors are selected from a group comprising: a camera, a LIDAR sensor, a RADAR sensor, a thermal imaging camera, and an infra-red (IR) camera (radar devices 120a-120h – see at least Fig. 1 and ¶ [0015]; cameras 140a-140d – see at least Fig. 1 and ¶ [0015]).
Regarding claim 3, Kriel further teaches:
configured to perform an object detection algorithm to determine, from the image data from each imaging sensor, an identity for the object (discriminator 220 classifies the detected object – see at least Fig. 2 and ¶ [0028]).
Regarding claim 4, Kriel further teaches:
wherein the user interface comprises a display screen (processor executes instructions to render graphics and images on display 260, e.g., image 420, in response to the alert – see at least ¶ [0042]).
Regarding claim 5, Kriel further teaches:
wherein the display screen comprises: a display terminal in an operator cab of the machine; and/or a screen of a remote user device (display 260 disposed in close proximity to the cabin of machine 110 and within view of the operator – see at least ¶ [0042]).
Regarding claim 6, Kriel further teaches:
wherein determining the configuration for the user interface comprises selecting from a group of possible display configurations in dependence on the determined identities (display 260 may provide indication boxes 430, 440 based on the type of object).
Regarding claim 7, Kriel further teaches:
wherein each of the configurations for the user interface comprises a representation of image data obtained by one or more of the imaging sensors (image 420 may be an image captured by camera 140 – see at least ¶ [0043]).
Regarding claim 11, Kriel further teaches:
configured to compare the determined identities for each of the imaging sensors (discriminator 220 compares metadata and payload of the detected object data to classification models – see at least ¶ [0029]) and determine the user interface configuration in dependence on the comparison (display a first alert or second alert based on type of object – see at least ¶ [0040]; display 260 may provide indication boxes 430, 440 based on the type of object).
Regarding claim 18, Kriel further teaches:
the control system of any preceding claim (see claim 1 rejection above) and a plurality of imaging sensors (radar devices 120a-120h – see at least Fig. 1 and ¶ [0015]; cameras 140a-140d – see at least Fig. 1 and ¶ [0015]).
Regarding claim 19, Kriel further teaches:
An agricultural vehicle (machine 110 may embody a fixed or mobile machine that performs some type of operation associated with an industry such as mining, construction, farming, transportation, or any other industry known in the art – see at least ¶ [0015]) comprising or being controllable by the control system of claim 18 (see rejection of claim 18).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Kriel in view of Kriel et al. (US 2014/0204215 A1, “Kriel II”).
Regarding claim 8, Kriel fails to teach but Kriel II discloses image processing using unified images and teaches:
wherein one or more of the configurations for the user interface comprises a representation of image data obtained by two or more of the imaging sensors (image processor 255 may combine image data captured by cameras 140a-140d into a unified image for rendering the unified image on display 260 – see at least Fig. 2 and ¶ [0031]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the object recognition system of Kriel to provide a representation of image data obtained by two or more imaging sensors, as taught by Kriel II, with a reasonable expectation of success, because it would provide for a single image in which the operator may focus attention instead of multiple different images displayed concurrently (Kriel II at ¶ [0003]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Kriel in view of Bingle et al. (US 9,623,799 B2, “Bingle”).
Regarding claim 9, Kriel fails to teach but Bingle discloses a camera module for a vehicle and teaches:
configured to determine the configuration for the user interface in dependence on an ambient light condition (imaging sensor or camera may be switched or set to a color mode based on ambient light levels – see at least 28:9-27).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the object recognition system of Kriel to provide for determining the configuration based on ambient light, as taught by Bingle, with a reasonable expectation of success, because it would provide for capability for low light imaging (Bingle at 1:22-28).
Regarding claim 10, Bingle further teaches:
wherein the ambient light condition is inferred from a time of day; and/or wherein an ambient light condition is determined from sensor data received from a light sensor on or otherwise associated with the agricultural machine (ambient light conditions detected by ambient light sensor – see at least 28:9-27).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combined object recognition system of Kriel and Bingle to provide for determining ambient light from a light sensor, as further taught by Bingle, with a reasonable expectation of success, because it would provide for capability for low light imaging (Bingle at 1:22-28).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kriel in view of Swaminathan et al. (US 10,140,855 B1, “Swaminathan”).
Regarding claim 12, Kriel further teaches:
wherein the imaging sensors comprise a camera (radar devices 120a-120h – see at least Fig. 1 and ¶ [0015]; cameras 140a-140d – see at least Fig. 1 and ¶ [0015]).
Kriel fails to teach the sensors comprise LIDAR sensor and a thermal camera.
However, Swaminathan discloses fusing multiple sensor data and teaches:
wherein the imaging sensors comprise a camera, a LIDAR sensor and a thermal camera (sensor data obtained from one or more sensors 120, which may include radar systems 121, video systems 122 such as cameras and thermal cameras, magnetometers 123, acoustic sensors 124, inductive loops 125, and any other devices or systems 126 which are capable of detecting a presence of objects 102 within a traffic environment, for example light-based (such as ultraviolet, visible, or infrared light) or laser-based sensing systems, such as LiDAR – see at least 3:28-46; any combination of such sensors may be used to detect objects – see at least 3:28-46; fused images to detect and classify objects – see at least abstract and 2:6-19).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the object recognition system of Kriel to provide a camera, LIDAR sensor, and thermal sensor, as taught by Swaminathan, with a reasonable expectation of success, because it would provide for more accurately classifying objects (Swaminathan at 2:6-19).
Allowable Subject Matter
Claims 13-17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON L TROOST whose telephone number is (571)270-5779. The examiner can normally be reached Mon-Fri 7:30am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached at 313-446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON L TROOST/Primary Examiner, Art Unit 3666
Prosecution Insights