DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Status of Claims
The amendment filed on 12/8/2025 has been entered. In the amendment, Applicant amended claims 1-2 and 5-8. Currently claims 1-8 are pending.
Claim Objections
Claim 1 is objected to because of the following informalities: “posture of a work machine” should be changed to “posture of [[a]]the work machine”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Rikkola et al. (US 2015/0111184) in view of Kato et al. (WO 2024/209665, machine translation of which is used for this examination).
Regarding claim 1, Rikkola teaches a display system (Fig. 1: computing device 10 including at least one monitor or screen as output device 22; [0023]: “The output device 22 can include at least one monitor or screen (e.g., a liquid crystal display ("LCD") monitor) that displays the generated simulated training environment to the operator. In some embodiments, the output device 22 includes multiple screens that provide the operator with a wide view of the training environment”) for a work machine ([0003]-[0009]: “industry equipment” includes a variety of work machine), comprising:
a hardware processor (Fig. 1: processing unit 12) configured to:
accept input of first operation data from a first operator (Fig. 1: input via input device 20; Fig. 5: step 110; [0007]: “The training simulator application is also configured to receive a second series of operating commands from the second operator for moving the simulated industrial equipment, and automatically modify the indicator based on the second series of operating commands”; [0031]: “the application 19 receives a series of operating commands from the trainee for the simulated industrial equipment (at block 110)”, “the trainee operating the simulated industrial equipment within the simulated working environment can attempt to duplicate or follow the optimal path by aligning the at least one component with the representation during the dig cycle”; Examiner’s Note: “a first operator” is interpreted as the second operator who is a trainee),
simulate a change in posture of the work machine in accordance with the first operation data ([0031]: “For example, if the trainee has moved the simulated dipper too far left as compared to the optimal digging path, the application 19 can set a direction indicator to a right arrow that informs the trainee that he or she should move the dipper to the right to keep the motion of the dipper aligned with an optimal digging path” implies a simulated change in posture of dipper has been obtained),
reproduce an operation performed by a second operator, on the basis of model data including second operation data previously input by the second operator ([0006]: “Accordingly, embodiments of the invention provide methods and systems for training an operator by establishing an optimal path of motion for at least one component of simulated industrial equipment provided within a training simulator (e.g., an optimal path of motion of a simulated dipper or a simulated bucket during a dig cycle). To establish the optimal path of motion, an operator experienced in operating the equipment (i.e., in the real world) that is simulated in the training simulator operates the training simulator one or more times. Data representing the operator's path of motion of the at least one component performed within the training simulator is stored. For example, data representing a complete dig cycle performed by the experienced operator (e.g., motion and speed) can be stored. The stored data is used to generate an optimal path of motion for the at least one component”; [0007]: “The training simulator application, when executed by the processing unit, is configured to generate a simulated working environment and simulated industrial equipment, receive a first series of operating commands from a first operator for moving at least a portion of the simulated industrial equipment, and calculate an optimal path based on the first series of operating commands”; [0008]: “The method includes generating a simulated mining environment and a simulated shovel including a simulated dipper, receiving a first series of operating commands from a first operator for moving the simulated dipper, and calculating, with a processing unit, an optimal digging path based on the first series of operating commands. The instructions also include generating a graphical representation of at least a portion of the optimal digging path and displaying the graphical representation to a second operator within the simulated mining environment”; Examiner’s Note: “second operator” is interpreted as first/experienced operator; “model data” is interpreted as stored data obtained from experienced operator’s operation used to generate optimal path to be used as a guide to train a trainee), and
(Figs. 2-4 and 6-13: simulated shovel 50, which includes a simulated dipper 55) representing the simulated change in the posture of the work machine
Rikkola does not further teach displaying a second moving image representing the reproduced operation performed by the second operator simultaneously with the first moving image.
Rikkola’s technique is different from the instant invention in that the former relies on use of indicators (Figs. 6-13) to guide the first/trainee operator while the latter displays both the operation of the trainee and the operation of the experienced operator on the same screen to guide the trainee. The differentiating limitation is not new, however.
Kato, for instance, teaches in Fig. 8 displaying both an experienced operator’s operation and an trainee operator’s operation on the same screen as an guide to train the trainee operator.
Before the effective filing date of the invention, it would have been obvious for one ordinary skill in the art to modify Rikkola’s technique with Kato’s technique displaying simultaneously both the first moving image association with the operation of the trainee and econd moving image association with the operation of the experienced operator on the same screen to guide the trainee to improve his/her performance with more intuitiveness compared to the use of indicators..
Regarding claim 2, Rikkola in view of Kato further teach the display system according to claim 1, wherein the hardware processor is configured to:
reproduce a behavior of the work machine on the basis of the model data (Kato: Fig. 8),
wherein the second moving image includes the reproduced operation performed by the second operator and the reproduced behavior of the work machine (Kato: Fig. 8), and
the first moving image and the second moving image are displayed in an overlaid manner (Kato: Fig. 8).
Regarding claim 7, Rikkola in view of Kato does not teach, in this embodiment, the display system according to claim 1, wherein
the first moving image and the second moving image are displayed on a head-mounted display, and
the second moving image is positioned forward in a gaze direction, on the head-mounted display.
Kato, however, additionally teaches in Figs. 8-9 wherein
the first moving image and the second moving image are displayed on a head-mounted display (Figs.8-9: HMD 29 for displaying an image of the trainee H's field of vision of a vehicle traveling on an experience course in accordance with the trainee H's simulated driving movements), and
the second moving image is positioned forward in a gaze direction (Page 3, 2nd paragraph: “The HMD 29 also includes a gaze detection sensor that detects the direction of the trainee H's gaze”; Page 7, 2nd paragraph: “the driving training processing unit 61 transmits the selected model driving data to the HMD driving unit 29b, and causes the HMD 29 to display an image of the field of view of the trainee H while driving the training course. The HMD driving unit 29b changes the image of the field of view according to the orientation and position of the HMD 29 detected by the angular velocity sensor 29a”), on the head-mounted display.
Before the effective filing date of the invention, it would have been obvious for one ordinary skill in the art to modify the technique of Rikkola in view of Kato in claim 1 with Kato’s additional teaching to enhance the trainee’s experience.
Claim 8 is rejected for substantially the same rationale as applied to claim 1.
Allowable Subject Matter
Claims 3-6 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 3: the claimed reference position image in the second moving image is specifically “indicating a reference position of an operation lever”.
Claim 6: “the second moving image represents a change in a position of the operation lever”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2018/0247558 by Livneh teaches in Fig. 1 aircraft 102 is a real aircraft operated by the operator (i.e., a pilot) and Clone 104 is a virtual clone of aircraft 102 generated according to the recorded state or at minimum to the recorded at least one parameter of aircraft 102, recorded during a previous activity session and operating according to the doctrine or profile.
US Patent No. 11,164,476 by Pittman teaches a heavy equipment simulation system for simulating an operation of a heavy equipment vehicle in a virtual environment, which includes a display device assembly configure to display the virtual environment including the simulated heavy equipment vehicle, a virtual reality (VR) headset unit adapted to be worn by the user, and a control system.
US 20150376868 by Jackson teaches a method for providing guidance to a construction machine operator, comprising: computing kinematic behavior of a construction machine based on data received from the construction machine while performing a task; comparing the kinematic behavior with a model for operating the construction machine for performing the task; and sending feedback data to the construction machine based on the comparing to provide guidance to the user of the construction machine for performing the task.
US 2015/0199106 by Johnson teaches a display system for use at a work site, which includes a display device having a display screen and a pose sensor system associated with the display device for generating display device pose signals indicative of a position and an orientation of the display device relative to the work site.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUEMEI ZHENG whose telephone number is (571)272-1434. The examiner can normally be reached Monday-Friday: 9:30 pm-6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benjamin Lee can be reached at 571-272-2963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XUEMEI ZHENG/Primary Examiner, Art Unit 2629
Prosecution Insights