Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
1. This communication is a first office action, non-final rejection on the merits. Claims 1-10, as originally filed, are currently pending and have been considered below.
Priority
2. The application is filed on 02/27/2025 but claims the benefit of foreign application number JP 2024-037494 filed on 03/11/2024.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 02/27/2025 has been considered. The submission is in compliance with the provisions of 37 CFR 1.97. Form PTO-1449 is signed and attached hereto.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
6. Claims 1-3 and 5-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over KIYOTA (WO 2014148204 A1) (hereinafter KIYOTA) in view of IZUMIKAWA (US 20220074171 A1) (hereinafter IZUMIKAWA).
Regarding claim 1, KIYOTA discloses a work machine (Page 1, lines 15-16, work machine peripheral monitoring transmitting an appearance of a person present around a work machine to an operator of the work machine, FIG. 2 showing shovel 60 as a working machine ) comprising:
a lower traveling body (page 2, lines 19-21, Fig. 2, shovel 60 is an upper portion of a crawler type lower traveling body 61 via a turning mechanism 62.); an
upper turning body mounted on the lower traveling body so as to freely turn (Fig. 2, lines 23-24, The upper swing body 63 also has a cab (driver's cab) 64);
a sound collecting device provided on the upper turning body so as to be able to detect a direction of a sound around the work machine (page 2, lines 12-15, work machine peripheral monitoring device that monitors periphery of a work machine, and includes control unit 1, camera 2, input unit 3, display unit 5, human detection sensor 6, an alarm, output unit 7 and sound collecting unit 8, page 1, lines 15-19, work machine peripheral monitoring device capable of transmitting an appearance of a person present around work machine to operator of work machine, localize sound source position and direction of sound source position, page 6, lines 17-26, control unit 13 the alarm output unit 7 based on determination result of the person presence / absence, sound collected by sound collection unit 8 to operator of the shovel 60); and
a control device configured to report a direction of speech based on the direction of the sound detected by the sound collecting device, when a person existing around the work machine is speaking (Fig. 1, monitors periphery of a work machine, control unit 1, a camera 2, display unit 5, human detection sensor 6, image generation unit 11, person existence / non-existence determination unit 12, alarm control unit 13 and voice transmission support unit 14, page 1, lines 15-19, work machine peripheral monitoring device capable of transmitting an appearance of a person present around work machine to operator of work machine, localize sound source position and direction of sound source position, page 6, lines 17-26, control unit 13 the alarm output unit 7 based on determination result of the person presence / absence, sound collected by sound collection unit 8 to operator of the shovel 60, extracts human voice from sound collected, page 24, lines 38-47, sound signal output and extracts sound signal related to human sound, transmit voice of worker working in the space outside the cab 64 to operator in the cab 64 isolated from outside space).
Even though KIYOTA discloses a lower traveling body (page 2, lines 19-21, Fig. 2, shovel 60 is an upper portion of a crawler type lower traveling body 61 via a turning mechanism 62.); an upper turning body mounted on the lower traveling body Fig. 2, lines 23-24, The upper swing body 63 also has a cab (driver's cab) 64) so as to freely turn KIYOTA specifically fails to disclose upper turning body mounted on the lower traveling body so as to freely turn.
In analogous art, IZUMIKAWA discloses upper turning body mounted on the lower traveling body so as to freely turn Fig.1, para 194, lower traveling body 1 moves forward while moving direction of lower traveling body 1 coincides with orientation of the upper turning body 3, operator in the cabin 10 can visually determine whether a worker in front of the crawlers 1C is located, para 272, control part 305 determine whether travel operation of lower traveling body 1 corresponds to straight line movement, power turn, a pivot turn, or a spin turn, based on operation information received input from operation information output device 29, para 112, The sound output device 54 outputs a sound toward at least one or both of interior of cabin 10 and exterior of the cabin 10).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify teaching of work machine peripheral monitoring device capable of transmitting an appearance of a person present around a work machine to an operator of the work machine disclosed by KIYOTA to display device displays an image representing that the predetermined object is detected outside a range of the displayed captured image of the first camera as taught by IZUMIKAWA to provide a technology that allows an operator of a shovel to readily identify the location of an object even when the object is detected outside a range of a peripheral image displayed on a display device [IZUMIKAWA, paragraph 032].
Regarding claim 2, KIYOTA discloses the work machine according to claim 1, wherein the control device reports the direction of speech by one or more of methods including displaying the direction of speech on a display device; outputting the sound from a sound output device corresponding to the direction of speech among a plurality of the sound output devices provided inside a driving room; lighting a light source corresponding to the direction of speech among a plurality of the light sources; and vibrating an operation device corresponding to the direction of speech among a plurality of the operation devices (page 2, lines 27-29, display part 5 is installed in the position in which the operator in the cab 64 visually recognizes, Page 3, lines 30-35, alarm output unit 7 is an alarm device that outputs sound and light, and includes a sound output device such as a buzzer and a speaker, and a light emitting device such as an LED and a flash light, and outputting an alarm sound, page 24, lines 9-15, apparatus 100 outputs an alarm sound from the alarm output unit 7, outputs light from alarm output unit 7, light contents (colors, etc.) ).
IZUMIKAWA also teaches shovel 100 recognizes speech spoken by a worker around the shovel 100, a gesture performed by a worker, and the like through a speech input device [043] and speech input device includes, for example, a microphone configured to obtain speech data spoken by the terminal apparatus user [085].
Regarding claim 3, KIYOTA discloses the work machine according to claim 2, wherein the control device displays information indicating the direction of speech on the display device in a display mode that is different from that of information indicating another direction, with reference to display information indicating the work machine (Fig. 25, page lines 5-9, transmission of speech collected by sound collection unit 8 to the operator of the shovel 60, Page 26, lines 48-50-page 27, lines 1-4, device 100 provided with an indicator indicating a voice recognition state at a position where a person present around the shovel 60 visually recognize, speech recognition state, a state in which a voice generated by a person present in the vicinity of shovel 60 is extracted by voice transmission support means 14 and a voice generated by a person present in the vicinity of the shovel 60 is a voice transmission support means 14, by looking indicator, a person present around shovel 60 can judge whether sound emitted by person is transmitted to operator of shovel 60 or not).
IZUMIKAWA also teaches if monitoring target is detected outside the surrounding area of the shovel 100, which corresponds to a monitoring image displayed on the display device 50, the controller 30 may notify the operator or the like that the monitoring target is detected by causing the sound output device 54 to output a predetermined sound or speech [0300].
Regarding claim 5, KIYOTA discloses the work machine according to claim 2, wherein the upper turning body is further provided with a plurality of imaging devices, and the control device displays image information captured by the plurality of imaging devices on the display device with reference to the work machine, in such a manner that the image information capturing the direction of speech is displayed in a display mode different from that of the image information of another direction (page 2, lines 27-29, display part 5 is installed in the position in which the operator in the cab 64 visually recognizes, Fig. 25, page lines 5-9, transmission of speech collected by sound collection unit 8 to the operator of the shovel 60, Page 26, lines 48-50-page 27, lines 1-4, device 100 provided with an indicator indicating a voice recognition state at a position where a person present around the shovel 60 visually recognize, speech recognition state, state in which a voice generated by a person present in the vicinity of shovel 60).
IZUMIKAWA discloses displays image indicating a surrounding area of the work machine from above the work machine based on image information captured by the imaging device (Fig. 6-7, 10, 20-22) and display processing part 301, the safety function control part 305 may superimpose and display a frame that surrounds the monitoring target appearing in the monitoring image, or may superimpose and display a marker at a position in the monitoring image, which corresponds to the actual position of the detected monitoring targe [0147] and image, having an isosceles triangle shape (or an equilateral triangle shape) whose apex faces upward, may be superimposed and displayed at the lower end of the monitoring image 910 (image captured by the camera 40B), and para 292, instruction image 913, the arrow shape or the isosceles triangle shape of such an instruction image may have gaps through which an image portion, on which the instruction image is superimposed, is exposed.
Regarding claim 6, KIYOTA discloses the work machine according to claim 1, wherein the upper turning body is further provided with an imaging device, and the control device corrects the direction of speech based on a direction of a person detected from image information captured by the imaging device (page 4, lines 11-24, camera captures ground surface from above, and an input image including an image in direction (a sky part) due to its wide angle of view. An image suitable for image conversion processing, generated from More specifically, the input image is projected onto a predetermined spatial model, The "space model" is a projection target of the input image. Specifically, the “space model” is configured by one or more planes or curved surfaces, Page 26, lines 48-50-page 27, lines 1-4, device 100 provided with an indicator indicating a voice recognition state at a position where a person present around the shovel 60 visually recognize, speech recognition state).
Regarding claim 7, KIYOTA discloses the work machine according to claim 1, wherein the upper turning body is further provided with an imaging device, and the control device prevents reporting of the direction of speech when a direction of a person detected from image information captured by the imaging device is different from the direction of speech (Page 26, lines 48-50-page 27, lines 1-4, device 100 provided with an indicator indicating a voice recognition state at a position where a person present around shovel 60 visually recognize, speech recognition state, page 1, lines 12-13, The imaging generation device 100 of a work machine peripheral monitoring device that monitors periphery of a work machine, page 24, lines 14-17, The light emission intervals etc. made different. This is to allow operator of shovel 60 to intuitively recognize rough position of person present around shovel 60 due to difference in the content of alarm, FIG. 13, showing the difference between output image shown in FIG. 12 and output image obtained by applying image loss prevention process to output image of FIG. 12).
Regarding claim 8, KIYOTA discloses the work machine according to claim 1, wherein a plurality of the sound collecting devices are provided, and each of the plurality of sound collecting devices is provided at a different position of the upper turning body so as to be able to detect the direction of the sound around the work machine (page 2, lines 14-18, The output unit 7 and sound collecting unit 8 are provided, image generating apparatus 100 generates an output image based on an input image captured by camera 2 mounted on a work machine, page 3, lines 40-45, The sound collection unit 8 is a device collects sound around the shovel 60, sound collection unit 8 is attached to surface collect, sounds emitted in a blind spot of the operator in the cab 64).
IZUMIKAWA discloses space model and re-projecting the projected images projected on the space model onto a different two-dimensional plane, space model is an object onto which a captured image is projected in a virtual space, and composed of one or more plane surfaces or curved surfaces that include a plane surface or a curved surface different from a plane surface in which captured image is positioned [128].
Regarding claim 9, KIYOTA discloses the work machine according to claim 8, wherein the control device prevents a sound generated from the work machine based on a sound signal input from the plurality of sound collecting devices (page 3, lines 48-49, The sound collecting unit 8 attached to side surface, page 6, lines 21-26, The sound transmission support means 14 is a means for supporting transmission of sound collected by sound collection unit 8 to operator of shovel 60, voice transmission support means 14 extracts human voice from sound collected by sound collection unit 8 ).
Regarding claim 10, KIYOTA discloses a control system for a work machine (Page 1, lines 15-16, work machine peripheral monitoring transmitting an appearance of a person present around a work machine to an operator of the work machine, FIG. 2 is a view showing a shovel 60 as a working machine) comprising:
the work machine including: a lower traveling body (page 2, lines 19-21, Fig. 2, shovel 60 is an upper portion of a crawler type lower traveling body 61 via a turning mechanism 62);
an upper turning body mounted on the lower traveling body so as to freely turn (Fig. 2, lines 23-24, The upper swing body 63 also has a cab (driver's cab) 64); and
a sound collecting device provided on the upper turning body so as to be able to detect a direction of a sound around the work machine (page 2, lines 12-15, work machine peripheral monitoring device that monitors periphery of a work machine, and includes control unit 1, camera 2, input unit 3, display unit 5, human detection sensor 6, an alarm, output unit 7 and sound collecting unit 8, page 1, lines 15-19, work machine peripheral monitoring device capable of transmitting an appearance of a person present around work machine to operator of work machine, localize sound source position and direction of sound source position, page 6, lines 17-26, control unit 13 the alarm output unit 7 based on determination result of the person presence / absence, sound collected by sound collection unit 8 to operator of the shovel 60), and
a control device configured to report a direction of speech based on the direction of the sound detected by the sound collecting device, when a person existing around the work machine is speaking (Fig. 1, monitors periphery of a work machine, control unit 1, a camera 2, display unit 5, human detection sensor 6, image generation unit 11, person existence / non-existence determination unit 12, alarm control unit 13 and voice transmission support unit 14, page 1, lines 15-19, work machine peripheral monitoring device capable of transmitting an appearance of a person present around work machine to operator of work machine, localize sound source position and direction of sound source position, page 6, lines 17-26, control unit 13 the alarm output unit 7 based on determination result of the person presence / absence, sound collected by sound collection unit 8 to operator of the shovel 60, extracts human voice from sound collected, page 24, lines 38-47, sound signal output and extracts sound signal related to human sound, transmit voice of worker working in the space outside the cab 64 to operator in the cab 64 isolated from outside space).
Even though KIYOTA discloses a lower traveling body (page 2, lines 19-21, Fig. 2, shovel 60 is an upper portion of a crawler type lower traveling body 61 via a turning mechanism 62.); an upper turning body mounted on the lower traveling body Fig. 2, lines 23-24, The upper swing body 63 also has a cab (driver's cab) 64) so as to freely turn KIYOTA specifically fails to disclose upper turning body mounted on the lower traveling body so as to freely turn.
In analogous art, IZUMIKAWA discloses upper turning body mounted on the lower traveling body so as to freely turn (Fig.1, para 194, lower traveling body 1 moves forward while moving direction of lower traveling body 1 coincides with orientation of the upper turning body 3, operator in the cabin 10 can visually determine whether a worker in front of the crawlers 1C is located, para 272, control part 305 determine whether travel operation of lower traveling body 1 corresponds to straight line movement, power turn, a pivot turn, or a spin turn, based on operation information received input from operation information output device 29, para 112, The sound output device 54 outputs a sound toward at least one or both of interior of cabin 10 and exterior of the cabin 10).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify teaching of work machine peripheral monitoring device capable of transmitting an appearance of a person present around a work machine to an operator of the work machine disclosed by KIYOTA to display device displays an image representing that the predetermined object is detected outside a range of the displayed captured image of the first camera as taught by IZUMIKAWA to provide a technology that allows an operator of a shovel to readily identify the location of an object even when the object is detected outside a range of a peripheral image displayed on a display device [IZUMIKAWA, paragraph 032].
7. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over KIYOTA (WO 2014148204 A1) (hereinafter KIYOTA) in view of IZUMIKAWA (US 20220074171 A1) (hereinafter IZUMIKAWA) and further in view of SHINKAI (US 20250019933 A1) (hereinafter SHINKAI).
Regarding claim 4, KIYOTA discloses the work machine according to claim 3, wherein the upper turning body is further provided with an imaging device, and displays the information indicating the direction of speech in the display mode that is different from that of the information indicating the other direction, with reference to the display information indicating the work machine included in the bird's-eye view image (Page 26, lines 48-50-page 27, lines 1-4, device 100 provided with indicator indicating a voice recognition state at position where person present around shovel 60 visually recognize, speech recognition state, state in which a voice generated by a person present in the vicinity of shovel 60 is extracted by voice transmission support means 14 and a voice generated by a person present in the vicinity of shovel 60 is a voice transmission support means 14, by looking indicator, person present around shovel 60 can judge).
KIYOTA and IZUMIKAWA fails to discloses displays a bird's-eye view image indicating a surrounding area of the work machine from above the work machine based on image information captured by the imaging device, with reference to the display information indicating the work machine included in the bird's-eye view image.
In analogous art, SHINKAI discloses displays a bird's-eye view image indicating a surrounding area of the work machine from above the work machine based on image information captured by the imaging device, with reference to the display information indicating the work machine included in the bird's-eye view image (para 75, The captured image may be, for example, a bird's eye image obtained by combining captured images captured by the rear camera S6B, the left camera S6L, and the right camera S6R, para 77, mage display part 41 displays a menu screen when the menu switch of the operation part 42 is operated in a state in which the bird's eye image FV and the rear image CBT captured by the imaging device S6 are displayed. For example, the image display part 41 reduces the size of the rear image CBT without changing the size of the bird's eye image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify teaching of work machine peripheral monitoring device capable of transmitting an appearance of a person present around a work machine to an operator of the work machine disclosed by KIYOTA and IZUMIKAWA to displays a menu screen when the menu switch of the operation unit 42 is operated while displaying overhead image captured by the imaging device as taught by SHINKAI to virtual viewpoint image generated by the control unit, and is generated based on images acquired by each of camera [SHINKAI, para 77].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mirza Alam whose telephone number is (469) 295-9286. The examiner can be reached on Monday-Thursday 7:30AM-6:00PM (EST).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Steven Lim can be reached on 571-270-1210. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for Published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MIRZA F ALAM/Primary Examiner, Art Unit 2688
Prosecution Insights