DETAILED ACTION
This is the First Office Action on the Merits and is directed towards claims 1-13 as originally presented and filed on 12/09/2024.
Notice of Pre-AIA or AIA Status
Priority is claimed as set forth below, accordingly the earliest effective filing date is May 26, 2023 (20230526).
The present application, effectively filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
This application claims the benefit of priority to Japanese Patent Application Nos. 2022-097599 and 2022-097600 filed on June 16, 2022 and is a Continuation Application of PCT Application No. PCT/JP2023/019734 filed on May 26, 2023 (20230526) (“Parent Application(s)”). See MPEP §201.07[R-08.2017]. In accordance with MPEP §609.02 [R-07.2015] Section A. 2 and MPEP §2001.06(b)[R-08.2017] (last paragraph), the Examiner has reviewed and considered the prior art cited in the Parent Application. Also in accordance with MPEP §2001.06(b) [R-08.2017] (last paragraph), all documents cited or considered ‘of record’ in the Parent Application are now considered cited or ‘of record’ in this application. Additionally, Applicant(s) are reminded that a listing of the information cited or ‘of record’ in the Parent Application need not be resubmitted in this application unless Applicants desire the information to be printed on a patent issuing from this application. See MPEP §609.02 [R-07.2015] Section A. 2. Finally, Applicants are reminded that the prosecution history of the Parent Application is relevant in this application. See e.g., Microsoft Corp. v. Multi-Tech Sys., Inc., 357 F.3d 1340, 1350, 69 USPQ2d 1815, 1823 (Fed. Cir. 2004) (holding that statements made in prosecution of one patent are relevant to the scope of all sibling patents).
Information Disclosure Statement
As required by M.P.E.P. 609 [R-07.2022], Applicant's 12/09/2024, 12/02/2025 and 03/24/2026 submission(s) of Information Disclosure Statement (IDS)(s) is/are acknowledged by the Examiner and the reference(s) cited therein has/have been considered in the examination of the claim(s) now pending. A copy of the submitted IDS(s) initialed and dated by the Examiner is/are attached to the instant Office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3 and 5, 8, 10, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over JP 2015164001 A to SUZUKI D (cited in the 12/09/2024 IDS) in view of US 20170339821 A1 to Ray; Brian Robert et al. (hereinafter Ray).
Regarding claim 1 SUZUKI teaches in for example the Figure(s) reproduced immediately below:
PNG
media_image1.png
312
406
media_image1.png
Greyscale
PNG
media_image2.png
406
333
media_image2.png
Greyscale
PNG
media_image3.png
633
451
media_image3.png
Greyscale
PNG
media_image4.png
514
359
media_image4.png
Greyscale
PNG
media_image5.png
590
408
media_image5.png
Greyscale
and associated descriptive texts a video display system comprising:
a camera attached to a work vehicle to which an implement is connected and configured to generate data of time-series images by performing imaging in a travel direction of the work vehicle (as shown in for example fig. 1, camera 32, work vehicle 10, etc. as explained in for example only:
“The camera 32 captures a field in front of the vehicle 10. The camera 32 includes a lens unit, an imaging unit including a CCD, an output unit for outputting a video signal, and the like. The camera 32 is attached slightly downward on the roof portion 13 of the vehicle 10. The lens portion of the camera 32 is disposed at a height h with respect to the ground G, and the optical axis of the lens portion forms a predetermined depression angle θ with respect to the horizontal direction. The camera 32 acquires an image in front of the agricultural machine 1 and outputs image information to the computer 35.”);
a screen (as shown in the figures above, and connotes display device 36 as explained in for example only :
“The display device 36 is display means for displaying the guide video output from the computer 35. For example, a liquid crystal display is used as the display 36. The input device 33 is a touch panel displayed on the display device 36, for example. When an instruction is input to the input device 33 by the operator, information indicating the instruction is output to the computer 35.”);
and a controller configured or programmed to display, on the screen, a video based on the data of the time-series images (as shown in the figures above and explained in for example:
“According to the guide device 30 of the agricultural working machine 1, the position information of the vehicle 10 is acquired by the GPS receiver 31. By the computing device 35b of the computer 35, the shape of the field where the vehicle 10 is located, the guide line indicating the route on which the vehicle 10 should travel, the travel locus of the vehicle 10, and the work area where the farm work has been performed by the vehicle 10 A guide image A1 indicating at least one is generated. Here, the captured image A2 captured by the camera 32 attached to the vehicle 10 is superimposed on the guide image A1 by the arithmetic device 35b of the computer 35 and displayed on the display 36 as AR (Augmented Reality). Is done. As shown in FIG. 5, the guide image A1 and the actual field shot image A2 are overlapped to display the guide image A3. Therefore, the information (for example, the guide line L, the travel locus X, etc.) shown in the guide image A1 is displayed. The relationship between the work area S, etc.) and the position of the crop C, structures, etc. can be grasped by the operator. Therefore, guide information according to the actual situation of the field can be provided.”);
wherein the controller is configured or programmed to, when the implement is connected to the work vehicle, display on the screen, a superimposed video in which a path for the implement is superimposed on the video (as shown in the figure 8 above and explained in for example:
“The superimposed video generation unit 47 generates a superimposed video in which the captured video A2 is superimposed on the guide image A1. On the other hand, the traveling information acquisition unit 48 acquires the traveling locus of the vehicle 10 (for example, the locus of the center point in the left-right direction of the vehicle 10) and the work area of the farm work already performed by the farm work machine 1. Then, the guide video generation unit 49 superimposes at least one of the traveling area and the work area on the superimposed video generated by the superimposed video generation unit 47 based on, for example, an input instruction via the input device 33.
The working unit that performs farm work in the farm work machine 1 is not limited to the case of extending from the vehicle 10 like the boom nozzle device 22. A mode in which the liquid or powder is ejected from an ejection port provided in a side portion of the vehicle 10 may be employed.”).
SUZUKI does not appear to expressly disclose wherein the controller is configured or programmed to, when the implement is connected to the work vehicle on an opposite side of the work vehicle from the travel direction, display on the screen, a superimposed video in which a path for the implement is superimposed on the video (emphasis added).
In analogous art Ray teaches in for example, the figures below:
PNG
media_image6.png
648
363
media_image6.png
Greyscale
PNG
media_image7.png
610
463
media_image7.png
Greyscale
PNG
media_image8.png
520
717
media_image8.png
Greyscale
PNG
media_image9.png
703
440
media_image9.png
Greyscale
And associated descriptive texts wherein the controller is configured or programmed to, when the implement is connected to the work vehicle on an opposite side of the work vehicle from the travel direction, display on the screen, a superimposed video in which a path for the implement is superimposed on the video (in for example Figures 1 and 6 wherein it is understood that item 104 connotes the path for the implement connected at the REAR of the tractor as explained in for example only paras:
“[0031] In the illustrated embodiment, the path visualization section 62 of the display 52 includes a graphical representation of the swath acquisition path 60 and a graphical representation of the guidance swath 80. The path visualization section 62 also includes a graphical representation of the agricultural vehicle 82. As illustrated, the graphical representation of the agricultural vehicle 82 is positioned remote from the graphical representation of the guidance swath 80, indicating that the agricultural vehicle is positioned remote from the guidance swath. In certain embodiments, the distance between the agricultural vehicle and the guidance swath may be manually entered (e.g., via a virtual numeric keypad) by depressing the “starting distance” virtual button 84. In addition, the initial speed of the agricultural vehicle (e.g., the speed of the agricultural vehicle before initiation of the swath acquisition process) may be manually entered (e.g., via a virtual numeric keypad) by depressing the “starting speed” virtual button 86. Furthermore, the angle between the initial agricultural vehicle path (e.g., the agricultural vehicle path before initiation of the swath acquisition process) and the guidance swath may be manually entered (e.g., via a virtual numeric keypad) by depressing the “starting angle” virtual button 88. Because the values may be manually entered, the operator may enter current values, expected values, test/example values, or a combination thereof. Furthermore, while the virtual buttons are located at the bottom of the path visualization section 62 of the display 52 in the illustrated embodiment, it should be appreciated that in alternative embodiments, the virtual buttons may be located in any other suitable portion of the display.
[0053] FIG. 6 is a diagram of an alternative embodiment of a display 52 that may be employed within a user interface of the control system of FIG. 2. In the illustrated embodiment, the controller is configured to determine a path of an agricultural implement coupled to the agricultural vehicle based at least in part on the swath acquisition path, and the controller is configured to output a signal to the display 52 indicative of instructions to present a graphical representation of the path of the agricultural implement 104. In certain embodiments, the controller may determine the path of the agricultural implement based at least in part on the type of agricultural implement, the configuration of the agricultural implement, the dimensions of the agricultural implement, the position and/or type of wheels/tracks on the agricultural implement (e.g., caster wheels/tracks, non-caster wheels/tracks, steerable wheels/tracks, etc.), the type/configuration of the hitch assembly coupling the agricultural implement to the agricultural vehicle, or a combination thereof, among other parameters. In the illustrated embodiment, the display 52 presents an “estimate implement path” checkbox that controls whether the agricultural implement path is displayed. In certain embodiments, the agricultural implement path may be automatically displayed if the controller detects the presence of an agricultural implement coupled to the agricultural vehicle. In such embodiments, the “estimate implement path” checkbox may be omitted.”).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the path planning of the implement disclosed in Ray with the path planning taught in SUZUKI with a reasonable expectation of success because it would have “made more efficient path planning of the implement” as taught by Ray Para(s):
“[0002] Certain agricultural vehicles, such as tractors or other prime movers, may be controlled by a control system (e.g., without operator input, with limited operator input, etc.) during certain phases of operation. For example, a controller may instruct a steering control system and/or a speed control system of the agricultural vehicle to automatically or semi-automatically guide the agricultural vehicle along a guidance swath within a field. In certain agricultural vehicles, an operator may manually direct (e.g., steer) the agricultural vehicle to the guidance swath and align the agricultural vehicle with the guidance swath before engaging the control system. Unfortunately, the process of acquiring the guidance swath manually may be inefficient, thereby increasing fuel consumption and/or delaying agricultural operations.”.
Regarding claim 2 and the limitation the video display system according to claim 1, wherein the implement is connected to a rear portion of the work vehicle;
the video shows an area in front of the work vehicle;
the controller is configured or programmed to display the video on the screen;
and the controller is configured or programmed to display, on the screen, the superimposed video in which the path for the implement is superimposed on the video (see the obviousness to combine and the rejection of corresponding parts of claim 1 above incorporated herein by reference wherein it is understood that the combination of both SUZUKI AND Ray teach the claimed limitations in for example Ray Fig. 6).
Regarding claim 3 and the limitation the video display system according to claim 1, wherein the controller is configured or programmed to predict the path for the implement based on implement information relating to the implement and pose information relating to a current pose of the work vehicle (See the teachings of Ray, fig. 6, wherein it is understood that given the Broadest Reasonable Interpretation (BRI) a Person of Ordinary Skill In The Art (POSITA) would see the path for the implement 104 is based on the pose of the vehicle which connotes path 60 and guidance swath 80 as explained in for example para:
“[0031] In the illustrated embodiment, the path visualization section 62 of the display 52 includes a graphical representation of the swath acquisition path 60 and a graphical representation of the guidance swath 80. The path visualization section 62 also includes a graphical representation of the agricultural vehicle 82. As illustrated, the graphical representation of the agricultural vehicle 82 is positioned remote from the graphical representation of the guidance swath 80, indicating that the agricultural vehicle is positioned remote from the guidance swath. In certain embodiments, the distance between the agricultural vehicle and the guidance swath may be manually entered (e.g., via a virtual numeric keypad) by depressing the “starting distance” virtual button 84. In addition, the initial speed of the agricultural vehicle (e.g., the speed of the agricultural vehicle before initiation of the swath acquisition process) may be manually entered (e.g., via a virtual numeric keypad) by depressing the “starting speed” virtual button 86. Furthermore, the angle between the initial agricultural vehicle path (e.g., the agricultural vehicle path before initiation of the swath acquisition process) and the guidance swath may be manually entered (e.g., via a virtual numeric keypad) by depressing the “starting angle” virtual button 88. Because the values may be manually entered, the operator may enter current values, expected values, test/example values, or a combination thereof. Furthermore, while the virtual buttons are located at the bottom of the path visualization section 62 of the display 52 in the illustrated embodiment, it should be appreciated that in alternative embodiments, the virtual buttons may be located in any other suitable portion of the display.”).
Although the claims are interpreted in light of the specification, limitations from the specification are NOT imported into the claims. The Examiner must give the claim language the Broadest Reasonable Interpretation (BRI) the claims allow.
See MPEP 2111.01 Plain Meaning [R-10.2024], which states
II. IT IS IMPROPER TO IMPORT CLAIM LIMITATIONS FROM THE SPECIFICATION
"Though understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim. For example, a particular embodiment appearing in the written description may not be read into a claim when the claim language is broader than the embodiment." Superguide Corp. v. DirecTV Enterprises, Inc., 358 F.3d 870, 875, 69 USPQ2d 1865, 1868 (Fed. Cir. 2004). See also Liebel-Flarsheim Co. v. Medrad Inc., 358 F.3d 898, 906, 69 USPQ2d 1801, 1807 (Fed. Cir. 2004) (discussing recent cases wherein the court expressly rejected the contention that if a patent describes only a single embodiment, the claims of the patent must be construed as being limited to that embodiment); E-Pass Techs., Inc. v. 3Com Corp., 343 F.3d 1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) ("Inter US-20100280751-A1 1pretation of descriptive statements in a patent’s written description is a difficult task, as an inherent tension exists as to whether a statement is a clear lexicographic definition or a description of a preferred embodiment. The problem is to interpret claims ‘in view of the specification’ without unnecessarily importing limitations from the specification into the claims."); Altiris Inc. v. Symantec Corp., 318 F.3d 1363, 1371, 65 USPQ2d 1865, 1869-70 (Fed. Cir. 2003) (Although the specification discussed only a single embodiment, the court held that it was improper to read a specific order of steps into method claims where, as a matter of logic or grammar, the language of the method claims did not impose a specific order on the performance of the method steps, and the specification did not directly or implicitly require a particular order). See also subsection IV., below. When an element is claimed using language falling under the scope of 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, 6th paragraph (often broadly referred to as means- (or step-) plus- function language), the specification must be consulted to determine the structure, material, or acts corresponding to the function recited in the claim, and the claimed element is construed as limited to the corresponding structure, material, or acts described in the specification and equivalents thereof. In re Donaldson, 16 F.3d 1189, 29 USPQ2d 1845 (Fed. Cir. 1994) (see MPEP § 2181- MPEP § 2186).
In Zletz, supra, the examiner and the Board had interpreted claims reading "normally solid polypropylene" and "normally solid polypropylene having a crystalline polypropylene content" as being limited to "normally solid linear high homopolymers of propylene which have a crystalline polypropylene content." The court ruled that limitations, not present in the claims, were improperly imported from the specification. See also In re Marosi, 710 F.2d 799, 802, 218 USPQ 289, 292 (Fed. Cir. 1983) ("'[C]laims are not to be read in a vacuum, and limitations therein are to be interpreted in light of the specification in giving them their ‘broadest reasonable interpretation.'" (quoting In re Okuzawa, 537 F.2d 545, 548, 190 USPQ 464, 466 (CCPA 1976)). The court looked to the specification to construe "essentially free of alkali metal" as including unavoidable levels of impurities but no more.).”
Regarding claim 5 and the limitation the video display system according to claim 3, wherein:
the work vehicle is capable of automatically traveling;
and the controller is configured or programmed to display, on the screen, the superimposed video in which a target line along a target path to be set to cause the work vehicle to automatically travel is further superimposed on the video (see the teachings of Suzuki Figs. 4 and 7 and Ray Fig. 6 and the obviousness to combine in the rejection of corresponding parts of claim 1 above incorporated herein by reference).
Regarding claim 8 and the limitation the video display system according to claim 3, wherein the controller is configured or programmed to predict the path for the implement including a predicted path for at least one of both end portions of the implement located in a width direction of the implement (see the figures of Suzuki which clearly displays where the ends of the paths meet, overlap or gap such as leakage region Sb in fig. 6).
PNG
media_image10.png
307
447
media_image10.png
Greyscale
Regarding claim 10 and the limitation the video display system according to claim 3, wherein the controller is configured or programmed to obtain the implement information including a size of the implement input by a user through an input interface (see the teachings of Ray para:
“[0029] FIG. 3 is a diagram of an embodiment of a display 52 that may be employed within the user interface of the control system of FIG. 2. In the illustrated embodiment, the display 52 is configured to present a visual representation of values of the swath acquisition parameters within a swath acquisition parameter section 58 and to present a graphical representation of the swath acquisition path 60 within a path visualization section 62 of the display 52. While the swath acquisition parameter section 58 is to the left of the path visualization section 62 in the illustrated embodiment, it should be appreciated that in alternative embodiments, the swath acquisition parameter section 58 and/or the path visualization section 62 may be in any suitable location on the display 52. In the illustrated embodiment, the display 52 is a touch sensitive display. Accordingly, the values of the swath acquisition parameters may be adjusted via interaction with the display 52. However, in alternative embodiments, the user interface may include other controls (e.g., switches, button, knobs, etc.) configured to enable an operator to adjust the values of the swath acquisition parameters.”).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the controls disclosed in Ray with the controls taught in Suzuki with a reasonable expectation of success because it would have “allowed the operator to adjust the values” as taught by Ray Para(s) [0029] above.
Regarding claim 13 and the limitation an agricultural machine comprising:
a work vehicle;
an implement;
and the video display system according to claim 1 (see the rejection of corresponding parts of claim 1 above incorporated herein by reference).
Claim 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over JP 2015164001 A to SUZUKI D (cited in the 12/09/2024 IDS) in view of US 20170339821 A1 to Ray; Brian Robert et al. (hereinafter Ray) as applied to the claims above in view of US 20220161722 A1 to COBURN M et al. (hereinafter Coburn).
Regarding claim 4 the combination of Suzuki and Ray does not appear to expressly disclose however in analogous art Coburn teaches in for example the figures below:
PNG
media_image11.png
817
608
media_image11.png
Greyscale
PNG
media_image12.png
452
617
media_image12.png
Greyscale
PNG
media_image13.png
673
507
media_image13.png
Greyscale
and associated descriptive texts the limitation a video display system wherein the controller is configured or programmed to:
predict a path for a wheel of the work vehicle based on the pose information when the work vehicle is traveling (in para:
“[0038] In some embodiments, the control circuitry may also generate for display predicted paths of the front wheels of the vehicle 204, 206 overlaid over the view of the environment outside of the vehicle from the selected view angle. For example, the control circuitry may generate for display two arrows 204, 206 as seen in FIG. 2, where left arrow 204 indicates a turn angle of the left front wheel and its predicted path over the environment outside of the vehicle, and where right arrow 206 indicates a turn angle of the right front wheel and its predicted path over the environment outside of the vehicle. This technique may be utilized, for example, when the driver of the vehicle is trying to navigate around and over rocks and boulders off-road. This may eliminate the need for a spotter outside the vehicle to provide instructions to the driver.”);
and display, on the screen, the superimposed video in which the path for the wheel of the work vehicle is further superimposed on the video (in Fig. 2 and para:
“[0037] FIG. 2 shows an exemplary illustration 200 of display 202 of a vehicle (e.g., one of vehicles 104, 134, 164 of FIGS. 1A-1C), in accordance with some embodiments of the present disclosure. In particular, FIG. 2 shows a view generated by the control circuitry on a heads-up display of a vehicle (e.g., one of displays 106, 136, 166). In some embodiments, the control circuitry generates for display a view of the environment outside of the vehicle (e.g., the rocky surface shown in FIG. 2) from the selected view angle and from the selected horizontal view position which were calculated according to techniques described above and below. For example, the view of the rocky surface may be a view generated by a virtual camera (e.g., one of virtual cameras 108, 138, 168) that is placed at the selected horizontal view position at the selected view angle. In some embodiments, the view generated for display on display 202 by the virtual camera may be the same or similar to a view that would be generated by a physical camera if it were physically positioned at the selected horizontal view position at the selected view angle.”).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the camera control system disclosed in Coburn with the display system taught in the combination of Suzuki and Ray with a reasonable expectation of success because it would have automatically “adjusted the view to suit the dynamic situation.” as taught by Coburn Para(s):
“[0002] A single camera of a vehicle can typically provide a view from only a single angle (e.g., a backup camera may provide only a single view angle). In one approach, a user may be allowed to manually select one of several views available from a camera (e.g., from a camera that has several selectable positions). However, when the user is engaged in driving a vehicle (e.g., driving the vehicle in off-road conditions), the user is typically unable to manually shift the camera view to fit the changing driving conditions. In addition, a single view from a camera may not provide the needed driving information. Consequently, what is needed is a camera control system that automatically adjusts the view to suit the dynamic situation.”.
Claims 6 and 7 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over JP 2015164001 A to SUZUKI D (cited in the 12/09/2024 IDS) in view of US 20170339821 A1 to Ray; Brian Robert et al. (hereinafter Ray) as applied to the claims above in view of US 20050090938 A1 to Ranelli, Ronald J.
Regarding claim 6 and the limitation the video display system according to claim 3, wherein the controller is configured or programmed to:
estimate a deviation amount of a predicted path for the implement from a reference path predicted from past tracks of the implement, based on the predicted path and the reference path;
and display different indications on the screen, depending on whether or not the deviation amount is less than a threshold (given the BRI connotes the figures shown in Suzuki especially fig. 4 as explained in for example:
“ Next, the comparison unit 45 compares the bird's-eye view with the captured video based on the video information. When the parameter is an initial value, the video of the camera 32 and the bird's eye view do not match due to an attachment error of the camera 32 or an individual difference of the cameras 32. Therefore, the comparison unit 45 captures the lattice pattern drawn on the ground G with the camera 32 and compares the captured image A2 with the guide image A1 based on the reference lattice data. Here, the grid pattern has a size and a shape corresponding to the reference grid data, and the comparison unit 45 compares the guide image A1 and the captured video A2 with a reference having the same size.
As shown in FIG. 4, the guide image A1 is shifted from the photographed video A2. More specifically, distortion B2 due to distortion of the camera 32 may occur at the peripheral edge between the guide image A1 and the captured video A2. Further, in the height and width of the display 36, a deviation B2 due to the viewing angle of the camera 32 may occur. In addition, a shift B3 due to the depression angle θ of the camera 32 may occur at the corner between the guide image A1 and the captured video A2. Further, a deviation B4 due to the ground height h of the camera 32 may occur at the side (end) of the guide image A1 and the captured video A2.
The comparison unit 45 detects these distortions or deviations and outputs a detection result. The parameter adjustment unit 46 changes the parameters so that the distortion or deviation detected by the comparison unit 45 is reduced, and generates the guide image A1 based on the changed parameters. In this way, the parameters are adjusted, and the guide image A1 is corrected so that the captured video A2 matches the guide image A1.“)
While the combination of Suzuki and Ray appears to teach the display while the deviation is within the threshold, the combination does not appear to expressly disclose what is displayed when the threshold is exceeded.
Ranelli teaches in for example the figures below:
PNG
media_image14.png
835
553
media_image14.png
Greyscale
generating a warning when a predicted path exceeds a threshold in for example step 132 as explained in for example para:
“[0037] Thus, if the comparison based upon the linear regression model warrants, a warning may be generated in step 132. The warning generated may be audible, visual, or a combination of the two. Also, as mentioned above with respect to FIG. 1, the audible and visual warnings may be external to the vehicle or internal to the vehicle. In step 134, should be the comparison predict a greater deviation from the predicted path such as beyond a threshold for generating warnings, driver intervention may be applied in step 134. As mentioned above, driver intervention may take the form of a driver intervention module 100 and various types of intervention due to the conditions may be provided. For example, the speed restriction, suspension center of gravity recovery, steering recovery, and gas tank load shifting may all be provided together, individually or in various combinations.”)
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the deviation warnings disclosed in Ranelli with the control system taught in the combination of Suzuki with a reasonable expectation of success because it would have provided the “vehicle operator an opportunity to correct the situation based on the conditions” as taught by Ranelli Para(s):
“[0004] In some situations, it may be desirable to provide the vehicle operator with an indication of a potential instability of the vehicle prior to losing control. This will provide the vehicle operator an opportunity to correct the situation based on the conditions.”.
Regarding claim 7 and the limitation the video display system according to claim 6, wherein the controller is configured or programmed to, when the deviation amount is not less than the threshold, display, on the screen, the superimposed video including a warning indication warning that the predicted path for the implement deviates from the past tracks (see the obviousness to combine and rejection of corresponding parts of claim 6 above incorporated herein by reference as well as Ranelli para:
“[0023] A visual indicator 90 such as a warning light, a navigation system display, or an instrument panel display may be controlled by a control signal from controller 14 in response to an unstable condition. Visual indicator 90 may also include an exterior visual indicator such as flashing headlamps.”)).
Regarding claim 11 and the limitation the video display system according to claim 9, further comprising:
a sensor to obtain sensing data indicating a distribution of objects on the ground around the work vehicle (given the BRI connotes the sensors in Suzuki);
wherein the controller is configured or programmed to:
estimate a size of an object existing on the ground in the travel direction of the work vehicle based on the sensing data output from the sensor (given the BRI connotes the collision imminence taught in Ranelli para:
“[0029] By utilizing an active comparison of where the vehicle is to where it should be (predicted path versus desired path), the interactive features of the system may provide the driver with a warning to tell the driver how fast the vehicle should be going and the best steering action while traveling. It is possible for the warning module to compare what the sensors are reading versus ideal sensor readouts for any given stretch of road. The warnings and interventions may then be tailored for the route, the topography, and the driver actions. Active warnings and intervention based on road conditions, collision imminence, pitch and roll readouts, speed, and over and under steering relative to the driver's current lane. The warnings and interventions may be initiated based on rate of change to a base position and then compared to the GPS location.
[0030] The system may also have a lag time and delay that would amount to a debouncing of signals and inputs prior to the initiation of driver warnings and intervention. The system may perform a constant comparison of an ideal trip versus the driver's interactions, along with speed, turning, lane drifting, collision avoidance, and upcoming road hazard feedback. The system in its comparison may use a predictive approach to driving, slowing down, curves ahead, object ahead warnings and allow timely intervention through driver actions.”);
and display, on the screen, the superimposed video including a warning indication warning that the implement is likely to hit the object on the ground, depending on a result of comparison between the size of the implement and the size of the object on the ground (see the teachings of the combination of references wherein it is understood that a POSITA would have found it obvious to prevent collisions by displaying warnings superimposed on the video so that the operator can immediately see where a collision may be predicted by either the tractor or the implement itself).
Claims 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over JP 2015164001 A to SUZUKI D (cited in the 12/09/2024 IDS) in view of US 20170339821 A1 to Ray; Brian Robert et al. (hereinafter Ray) as applied to the claims above in view of US 20200344939 A1 to Sporrer; Adam D. et al. (hereinafter Sporrer).
Regarding claim 9 the combination of Suzuki does not appear to expressly disclose the video display system wherein the controller is configured or programmed to obtain the implement information including a size of the implement by communicating with the implement.
In analogous art Sporrer teaches in for example, the figures below:
PNG
media_image15.png
508
697
media_image15.png
Greyscale
PNG
media_image16.png
729
523
media_image16.png
Greyscale
PNG
media_image17.png
707
522
media_image17.png
Greyscale
And associated descriptive texts a video display system wherein a controller is configured or programmed to obtain implement information including a size of the implement by communicating with the implement (as explained in for example paras:
“[0024] Once a residue coverage metric has been determined, the metric may be utilized to control aspects of a future operation over the field. For example, in an ongoing tillage operation, if residue metrics from a rearward image indicate insufficient residue coverage or size, various aspects of the tillage implement (e.g., disk or ripper depth) may be automatically adjusted in order to provide greater residue coverage or size. Similarly, if a comparison of residue metrics from forward and rearward images indicates that an ongoing tillage operation is decreasing residue coverage or size too aggressively, various aspects of the implement may be automatically adjusted accordingly. Additionally, if a comparison of residue metrics from lateral zones indicates that there is a lack of uniformity laterally across the rear of the implement, various aspects of the relevant implement may be automatically adjusted accordingly. For example, the angle of a disk gang may be adjusted to distribute residue from a highly concentrated zone to a lower concentrated zone to increase residue uniformity. Uniform residue coverage can be useful later for air seeders, planters, row cleaners, etc.
[0026] The computer system or device may be included on the relevant implement (e.g., as part of an embedded control system). In certain examples, the computer system or device may be included on another platform (e.g., a tractor towing the implement or a remote ground-station) and may communicate with various devices on the implement (e.g., various control devices) via various known means. In one example, the computer system or device may be in communication with a controller area network (CAN) bus associated with the implement or an associated vehicle, in order to send and receive relevant control and data signals.
[0050] In another example, a control system may have access to the dimensions of the implement and the locations of the camera assemblies 74 and 78 and the locations of the fields of view of the camera assemblies. In that way, based on the location of the implement 10, and as that location changes, the control system can determine when a rearward image is of the same portion of the field as a previously captured forward image. Thus, the two images can be corelated with one another based on a location where they were taken instead of, or in addition to, the time offset between the images and the speed and heading of the implement.”).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the communications disclosed in Sporrer with the system taught in the combination of Suzuki with a reasonable expectation of success because it would have “minimized variances in residue coverage” as taught by Sporrer Para(s):
“[0018] In this light, it may be useful to provide a system and method for actively assessing aspects of residue coverage during a particular operation and utilizing this assessment to control ongoing aspects of the particular operation or a different, future operation. For example, for a primary tillage (or other) operation, it may be useful to provide a control system that determines the percent coverage and characteristic size of residue on a portion of field that has already been tilled (or otherwise addressed), then utilize the determined percent coverage and characteristic size to guide the continuing tillage (or other) operation or a future operation (e.g., a secondary tillage operation or planting operation) on the same field. For more specialized control of the tillage operation, the areas rearward and forward of the tillage implement can be divided into a plurality of different zones. Each zone can be independently monitored for percent coverage, characteristic size or other metrics. This way, variances of uniformity in residue coverage can be monitored and addressed.”.
Claims 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over JP 2015164001 A to SUZUKI D (cited in the 12/09/2024 IDS) in view of US 20170339821 A1 to Ray; Brian Robert et al. (hereinafter Ray) as applied to the claims above in view of US 20220161722 A1 to COBURN M et al. (hereinafter Coburn) and US 20190075255 A1 to MATSUMOTO; Hideyuki et al. (hereinafter Matsumoto).
Regarding claim 12 and the limitation the video display system according to claim 1, further comprising:
a light source to be controlled by the controller;
and an optical system to receive light emitted from the light source and form a virtual image on the front of the screen (Ray teaches virtual items in for example paras:
“[0030] In the illustrated embodiment, the swath acquisition parameter section 58 includes an overshoot adjustment control 64, a swath turn rate control 66, an approach angle control 68, and an initial turn rate control 70. As illustrated, each control includes a virtual slider 72 configured to adjust the value of the respective parameter, and each control includes a numerical representation 74 of the value of the respective parameter. To adjust the value of each parameter, the respective slider 72 may be moved along a slider path. As the slider moves, the numerical representation 74 of the respective parameter changes based on the position of the slider 72 along the slider path. Each slider 72 may be moved by direct interaction, or by selecting an increase value arrow 76 or a decrease value arrow 78. While each control includes a slider and a numeric representation of the parameter value in the illustrated embodiment, it should be appreciated that in alternative embodiments, other control(s) and/or representation(s) of the parameter value may be displayed. For example, in certain embodiments, each control may include a virtual knob, a virtual dial, a field in which a number may be entered by a virtual keypad, or a combination thereof, among other input techniques. In addition, in certain embodiments, each control may include a graphical representation (e.g., a graph, a dial, etc.) of the value of the respective parameter. Furthermore, in certain embodiments, the parameter adjustment input/control may be positioned remote from the visual representation of the respective parameter.”;
Coburn teaches virtual images in for example paras:
“[0025] In some embodiments, one of the cameras of vehicle 104 may be a wide-angle camera. In this case, the control circuitry may modify the wide-angle output (e.g., by cropping and distortion techniques) to produce a view from the selected view angle. In some embodiments, vehicle 104 may have several cameras located at multiple positions and angles. In this case, the control circuitry may generate a virtual view from the selected view angle by combining outputs of the multiple cameras. The generated virtual view may then be displayed on the screen.
[0026] In some embodiments, the control circuitry may generate a view from a virtual camera. The term virtual camera may refer to any kind of camera object that is generated as software representation of a real camera that does not exist as a physical camera. For example, a virtual camera may be a software module that emulates operation of a real camera. In some embodiments, the virtual camera software module may emulate an output of a camera as if it were placed in a certain location. For example, the virtual camera software module may emulate an output of a real camera by combining output of real cameras and/or extrapolating the view based on the outputs of several real cameras. In some embodiments, the control circuitry may generate a virtual camera that emulates an output of a real physical camera, as if that real camera were placed in a position corresponding to the selected horizontal view position and oriented at a pitch corresponding to the selected view angle. Then, the control circuitry may use the virtual camera to generate a virtual view (e.g., by using outputs of multiple cameras) that is equivalent or similar to a view that would be produced if a real camera were placed in the position of the virtual camera. The generated virtual view may be displayed on a display of vehicle 104.”.
The combination of Suzuki does not appear to expressly disclose an optical system to receive light emitted from the light source and form a virtual image in front of the screen. (Emphasis added)
In analogous art Matsumoto teaches in for example, the figures below:
PNG
media_image18.png
481
781
media_image18.png
Greyscale
PNG
media_image19.png
606
493
media_image19.png
Greyscale
And associated descriptive texts and an optical system to receive light emitted from the light source and form a virtual image in front of a screen (as explained in for example para:
“[0030] FIG. 2 is a diagram showing an internal part of the vehicle 1 according to the first embodiment. FIG. 2 is diagram showing a driver's cabin inside the vehicle 1 seen from the driver's seat in the front direction of vehicle 1. The vehicle 1 includes a steering wheel 10, a dashboard 12, a windshield 14, a center console 16, a cluster panel 18 that displays the travelling speed and the engine speed of the vehicle etc. The center console 16 may further include a center display unit 20 that displays a navigation screen or the like. In the vehicle 1, a head-up display unit 22 on which video images are displayed by a head-up display may be provided on the upper part of the cluster panel 18. When the head-up display is a combiner type, the head-up display unit 22 is a combiner. Further, when the head-up display displays virtual images on the windshield 14, the head-up display unit 22 is an area of the windshield 14 where the virtual images are presented. Further, the vehicle 1 may include a rearview monitor 24. The rearview monitor 24 is arranged in a position similar to a position where a rearview mirror for checking the rear side of a vehicle in a typical vehicle, that is, at around the center of the upper part of the windshield 14.”).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the heads up display disclosed in Matsumoto with the display system taught in the combination of Suzuki with a reasonable expectation of success because it would have increased the safety of the vehicle by allowing the driver to keep their eyes in the direction of travel and see images/warnings/etc. overlaid on the windshield of the vehicle as taught by Matsumoto.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure as teaching, inter alia, the state of the art of work vehicle video systems at the time of the invention. For example:
US 20210185885 A1 to Sibley; Gabriel Thurston et al. teaches, inter alia a CALIBRATION OF SYSTEMS TO DELIVER AGRICULTURAL PROJECTILES in for example the ABSTRACT, Figures and/or Paragraphs below:
“Various embodiments relate generally to computer vision and automation to autonomously identify and deliver for application a treatment to an object among other objects, data science and data analysis, including machine learning, deep learning, and other disciplines of computer-based artificial intelligence to facilitate identification and treatment of objects, and robotics and mobility technologies to navigate a delivery system, more specifically, to an agricultural delivery system configured to identify and apply, for example, an agricultural treatment to an identified agricultural object. In some examples, a method may include identifying an emitter of an agricultural projectile delivery system to calibrate a trajectory of an agricultural projectile to intercept a target, predicting a projectile impact site relative to the reference of alignment, determining a calibration parameter to align the projectile impact site and the target, and adjusting the trajectory based on the one or more calibration parameters.
14. The method of claim 1, further comprising: implementing a first image capture device to generate a virtual target image of the target; implementing a second image capture device configured to form a portion of an image that is associated with an optical ray extending to the target, the optical ray including an occluded portion relative to the first image capture device; and implementing the focused light source as a coherent light source, the focused light source being configured to adjust a point on a surface at which the coherent light impinges to coincide with the projectile impact site.”.
US 20200354001 A1 to Hayashi; Hiroyuki teaches, inter alia a Work Vehicle in for example the ABSTRACT, Figures and/or Paragraphs below:
“A first controller of a work vehicle executes a first composition process for composition of a first video imaged by a camera and a second video imaged by a camera and generating composite video data indicating a composite video, and a transmission process for transmitting, to a second controller, the composite video data generated in the first composition process. The second controller of the work vehicle executes a reception process for receiving the composite video data from the first controller, and a display process for displaying the composite video indicated by the composite video data received in the reception process on a display.
PNG
media_image20.png
714
582
media_image20.png
Greyscale
[0067] FIG. 6 is a video displayed on the display 44 of the rough terrain crane 10 in FIG. 5. The video illustrated in FIG. 5 is obtained by composition of videos imaged by the cameras 61 to 64. More specifically, the video illustrated in FIG. 5 is a bird's-eye video obtained by composition of videos showing a situation in a bird's-eye view from above around the rough terrain crane 10. Hereinafter, operations of the first controller 51 and the second controller 53 for displaying the video illustrated in FIG. 5 on the display 44 will be described with reference to FIG. 7.”.
US 20210043085 A1 to Kreiling; Jeffery R. et al. teaches, inter alia a GUIDANCE DISPLAY SYSTEM FOR WORK VEHICLES AND WORK IMPLEMENTS in for example the ABSTRACT, Figures and/or Paragraphs below:
“A work vehicle guidance display system comprising: at least one imaging device disposed on a work vehicle; a display disposed in the work vehicle configured to display images from the imaging device; and a controller configured to: select a field of view of the imaging device to display; receive a static dimension associated with the work vehicle; receive a dynamic dimension associated with the work vehicle; and display on the display a field view with a first machine travel path based on the static dimension and a second machine travel path based on the dynamic dimension.
PNG
media_image21.png
516
668
media_image21.png
Greyscale
[0069] In the example shown in FIG. 4, the controller 48 of the implement guidance display system 22 further generates a baseline or “zero deviation” implement path graphic 136 on the HUD 96, as well as a zero deviation implement path graphic 138 on the HDD 102. The zero deviation implement path graphics 136, 138 each indicate the trajectory of the FEL bale spear attachment 28 when in a predetermined home orientation (the lowered, near-ground position) and traveling a straight (zero deviation) path in a forward direction. A visual contrast is thus created between the zero deviation implement path graphics 136, 138 and the projected implement path graphics 110, 120 to visually emphasize or highlight the degree to which the projected implement path graphics 110, 120 diverge from the zero deviation path, whether by turning of the loader 20 (as shown in FIG. 4) or by vertical movement of the FEL bale spear attachment 28 (as shown and discussed below in conjunction with FIG. 5). When generated on either or both of the HDD 102 and HUD 96, the zero deviation implement path graphics 136, 138 may be generated in a different color than the projected implement path graphics 110, 120, rendered partially transparent (e.g., to resemble a shadow or ghost image), or otherwise imparted with a varied appearance to provide contrast with the graphics 110, 120 and to avoid visually cluttering the depicted implement guidance displays.”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL LAWSON GREENE JR whose telephone number is (571)272-6876. The examiner can normally be reached on MON-THUR 7-5:30PM (EST).
Examiner interviews are available via telephone and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached on (571) 272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL L GREENE/Primary Examiner, Art Unit 3665 20260417
Prosecution Insights