DETAILED ACTION
Notice of Pre-AIA or AIA Status
In the present application, filed on or after March 16, 2013, claims 16-30 have been considered and examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 09/27/2024 is in compliance with the provision of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by Examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 16-17, 21, 24, and 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Schloss et al. (Schloss - US 2019/0348057 A1) in view of Cheng et al. (Cheng – Activity Analysis of Construction Equipment Using Audio Signals and Support Vector Machines), Gogolin (Gogolin – US 2015/0020609 A1), and Wisley et al. (Wisley – US 2020/0105072 A1).
As to claim 16, Schloss discloses a system installed in a mining truck that includes a dump body pivotally mounted on a frame, a cab where an operator of the mining truck sits, and an engine associated with the frame, the system implementing a method for detecting acoustic events denoting phases of a cycle of the mining truck, and the system comprising:
at least one acoustic device (Schloss: FIG. 1 the one or more sensors 42 and FIG. 2 the acoustic sensors 52) that captures an acoustic event (Schloss: Abstract, [0020], [0024], [0027]-[0028], [0045], and FIG. 1-2 the acoustic sensors 52: the work machine 20 includes one or more sensors 42 that are positioned in and around the work machine 20, such as but not limited to, acoustic sensors, vision sensors, accelerometers, vibration sensors, orientation sensors and the like), associated with a phase of a cycle of the mining truck (Schloss: [0002], [0017], [0023], [0045], and FIG. 2: the present disclosure may find application in many industries, including but not limited to, construction, mining, agriculture, and other such industries) and that generates one or more signals indicative of the captured acoustic event (Schloss: Abstract, [0020], [0024], [0027]-[0028], [0045], and FIG. 1-2 the acoustic sensors 52 In one non-limiting example, each acoustic signal detection array 56 is configured to monitor and detect work site noise 46 (e.g., machine noise, equipment noise, traffic noise), verbal communication 50 and other such noise present in and around the work site 33),
the acoustic device comprising:
at least one microphone that detects and captures the acoustic events associated with each phase of the mining cycle (Schloss: [0023]-[0024], [0027]-[0028], [0030]-[0031], and FIG. 1-2: ), the at least one microphone being mounted on or in the mining truck (Schloss: [0002], [0017], [0045], FIG. 1 the one or more sensors 42 and FIG. 2 the acoustic sensors 52: the sound monitoring and analysis system 60 includes a plurality of acoustic sensors 52 attached around the frame 24 or other such location of the work machine 20; however the acoustic sensors 52 may be positioned in other locations as needed. Each acoustic sensor 52 includes an acoustic signal detection array 56 formed using a plurality of microphones 54); and
an acoustic recording device associated with the at least one microphone that records the acoustic events captured by the at least one microphone (Schloss: [0020]-[0021], [0050]-[0052], and FIG. 6: The plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels. Furthermore, the sound monitoring and analysis system 60 may monitor noise produced by other machine components and systems, such as but not limited to the hydraulic actuating cylinders 40 (e.g., hydraulic pump stroke noise), the ground engaging elements 32 and the work tool 34);
at least one memory configured to store an application for analyzing the signals generated by the acoustic device (Schloss: [0020]-[0021], [0024]-[0025], [0030], [0050]-[0052], and FIG. 6: the machine control module 44 may serve as a centralized controller for the sound monitoring and analysis system 60, such that acoustic data (i.e., acoustic signals converted to digital signals) collected by each of the acoustic sensors is received by the machine control module 44 for further analysis, filtering, compression or other such operation) and representative of the phases executed by the mining truck with which the captured acoustic events are associated (Schloss: [0020]-[0021], [0050]-[0052], and FIG. 6: The plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels. Furthermore, the sound monitoring and analysis system 60 may monitor noise produced by other machine components and systems, such as but not limited to the hydraulic actuating cylinders 40 (e.g., hydraulic pump stroke noise), the ground engaging elements 32 and the work tool 34);
the one or more processors (Schloss: FIG. 1 the machine control module 44) comprising a module for executing the analyzing application, which processes the signals indicative of the acoustic events to determine a presence and/or absence of particular sounds and/or a frequency and duration of the particular sounds (Schloss: [0020]-[0021], [0050]-[0052], and FIG. 6: The plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels),
the one or more processors being capable of executing programmed instructions stored in memory to carry out the following steps: a step of operating the system, during which step the acoustic device captures one or more acoustic events in real time and generates one or more signals indicative of each captured acoustic event (Schloss: [0020]-[0021], [0050]-[0052], and FIG. 6: The plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels); and
a step of filtering and analyzing the sent signals, which step is performed by the at least one or more processors receiving the signals (Schloss: [0025], [0028], [0030]-[0033], [0047], [0050]-[0052], FIG. 1, and FIG. 6: Furthermore, the machine control module 44 may have one or more digital signal processors 78 (DSP) which further analyze, filter, compress or perform other such operations on the acoustic signal 58 detected by each microphone 54 and converted from an analog signal into a digital signal by the ADC 64).
Schloss does not explicitly disclose
each phase consisting of a number of specific acoustic events arranged in a predetermined chronological order,
one or more communication servers each comprising at least one or more processors operationally connected to the memory,
a step of sending the one or more signals to the one or more communication servers, which step is performed by the acoustic device;
a step of filtering and analyzing the sent signals, which step is performed by the at least one or more processors of the one or more communication servers receiving the signals; and
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases, during which step the at least one or more processors compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated,
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading and periods of driving loaded and empty to be identified from amplitude and frequency of the recordings.
However, it has been known in the art of audio-based analysis for heave equipment to implement each phase consisting of a number of specific acoustic events arranged in a predetermined chronological order,
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases, during which step the at least one or more processors compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated,
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading from amplitude and frequency of the recordings, as suggested by Cheng, which discloses each phase consisting of a number of specific acoustic events arranged in a predetermined chronological order (Cheng: Abstract, 2.4 Gaps in knowledge: why use sound: The audio pattern generated by each individual machine is often independent of the operator and the specific way that the task is performed. Operators can perform a task in several ways. For example, imagine a hydraulic excavator is digging a trench. This operation might include a series of movements such as diggings, rotating, swinging, and loading. These tasks could be handled in various ways such as different angles, swinging to left or right, etc. A computer vision algorithm would likely need to consider all these scenarios separately, while the audio signal analysis would always yield the same result),
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases (Cheng: FIG. 5-20), during which step the at least one or more processors compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated (Cheng: FIG. 5-20: Another way to visually evaluate the performance of the system is throughout the comparison charts as shown in FIG. 6, 8, 10, 12, 14, 16, 18, and 20. Tables 2-9 indicate that the performance of the proposed system for automatically recognizing activities of single machines is very promising. Generally, the proposed system can have over 80% even 85% accuracy identifying different activities. For JCB 3CX mini excavator and CAT 322C with hydraulic hammer, as shown in Tables 4 and 6, "Act 1" and "Act 2" have strongly different patterns in their STFTs, thus the identification accuracy can be over 90%. The proposed audio-based framework provides an efficient way to identify different activities for construction equipments. The machine learning algorithm needs only a few seconds of recording to obtain sufficient data to construct a model for learning audio patterns. We do not need a large database compared to neural network based machine learning method to implement the identification. Also, the machine learning model can learn the audio patterns for each activity without manual analysis of the audio recording),
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading from amplitude and frequency of the recordings (Cheng: 2.3 Action recognition of construction equipment using computer vision, 2.4 Gaps in knowledge: why use sound, and FIG. 5-20: The audio pattern generated by each individual machine is often independent of the operator and the specific way that the task is performed. Operators can perform a task in several ways. For example, imagine a hydraulic excavator is digging a trench. This operation might include a series of movements such as diggings, rotating, swinging, and loading. These tasks could be handled in various ways such as different angles, swinging to left or right, etc. A computer vision algorithm would likely need to consider all these scenarios separately, while the audio signal analysis would always yield the same result ).
Therefore, in view of teachings by Schloss and Cheng, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss to implement each phase consisting of a number of specific acoustic events arranged in a predetermined chronological order,
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases, during which step the at least one or more processors compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated,
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading from amplitude and frequency of the recordings, as suggested by Cheng. The motivation for this is to implement an audio based sensing information to monitor activities of a construction machine.
The combination of Schloss and Cheng does not explicitly disclose wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings.
However, it has been known in the art of audio-based analysis for heave equipment to implement wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings, as suggested by Gogolin, which discloses wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings (Gogolin: Abstract, [0021], [0038], and FIG. 1-2: audio signals from the influx of the load 104 may differ from those produced by the machine's engine, processes/operations running in the vicinity, and/or from other surrounding environmental sounds. Another factor of influx determination may be based upon the material of the load 104 (see FIG. 1). For example, a delivery of load 104 containing gravel, stones, and /or other materials, into an empty loading receptacle 106, may produce a specific sound, distinguishable from the sounds of other activity, objects, or events).
Therefore, in view of teachings by Schloss, Cheng, and Gogolin, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss and Cheng to implement wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings, as suggested by Gogolin. The motivation for this is to implement an audio based sensing information to monitor activities of a construction machine.
The combination of Schloss, Cheng, and Gogolin does not explicitly disclose one or more communication servers each comprising at least one or more processors operationally connected to the memory,
a step of sending the one or more signals to the one or more communication servers, which step is performed by the acoustic device; and
a step of filtering and analyzing the sent signals, which step is performed by the at least one or more processors of the one or more communication servers receiving the signals.
However, it has been known in the art of heave equipment to implement one or more communication servers each comprising at least one or more processors operationally connected to the memory,
a step of sending the one or more signals to the one or more communication servers, which step is performed by the device; and
a step of filtering and analyzing the sent signals, which step is performed by the at least one or more processors of the one or more communication servers receiving the signals, as suggested by Wisley, which disclose one or more communication servers each comprising at least one or more processors operationally connected to the memory (Wisley: [0058], [0076]-[0077], [0081]-[0099], and FIG. 1-3: The computer system 16 may store simulated machine operational data indicative of at least one optimal operating condition of the work machine 11),
a step of sending the one or more signals to the one or more communication servers, which step is performed by the device (Wisley: Abstract, [0042], [0046]-[0048], [0080], [0109], and FIG. 1-3: the computer system 16 may be configured to receive actual surface profile data, actual machine operating condition data and/or the actual route data and perform simulations of the operation of the at least one work machine 11. The computer system 16 may be separate from the work machine 11 and at least one surveying device 15 as illustrated (e.g. by being located in a separate housing) and they may communicate data within one another via the communication system 17. The computer system 16 may be located in a monitoring station on the worksite 14 or at a station remote to the worksite 14. For example, the computer system 16 may be located in a central server and database of the operating company of the worksite 14, the at least one surveying device 15 and/or the at least one work machine 11); and
a step of filtering and analyzing the sent signals, which step is performed by the at least one or more processors of the one or more communication servers receiving the signals (Wisley: [0058], [0076]-[0077], [0081]-[0099], and FIG. 1-3: At an analysis step 36 the monitored operating condition data may be processed in order to analyse the behaviour of the work machine 11 as it travels along the route 12. At least one type of analysis, as described below, may be carried out. The results of the analysis step 36 may be displayed to an operator on a display, such as in the form of at least one plot, gauge, map or table).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, and Wisley, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, and Gogolin to implement one or more communication servers each comprising at least one or more processors operationally connected to the memory,
a step of sending the one or more signals to the one or more communication servers, which step is performed by the device; and
a step of filtering and analyzing the sent signals, which step is performed by the at least one or more processors of the one or more communication servers receiving the signals, as suggested by Wisley. The motivation for this is to implement one or more processors at a server for remotely performing data analysis to determine operation conditions of a construction machine.
As to claim 17, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 16 further comprising the system according to claim 16, wherein the phases of the mining cycle incorporating the associated acoustic events that are detected and recorded comprise:
a phase of loading ore into the dump body while the mining truck is stationary in a loading area (Gogolin: [0021]-[0022], [0024]-[0026], [0029], [0038], and FIG. 1-2: Such audio files may be compared with the continuous rolling audio signal sample to determine an influx (or an outflow) of load 104. Effectively, the sound-processing module 206 may process the continuous rolling audio signal sample from ambient sound signals to generate a filtered audio sample. In that way, the sound-processing module 206 may determine a load influx or a load outflow. The memory may also store a minimum decibel threshold associated with each of these materials. Moreover, inflow and outflow audio signal templates may be stored as well );
a phase of driving the mining truck loaded (Gogolin: Abstract, [0021], [0038], and FIG. 1-2: audio signals from the influx of the load 104 may differ from those produced by the machine's engine, processes/operations running in the vicinity, and/or from other surrounding environmental sounds. Another factor of influx determination may be based upon the material of the load 104 (see FIG. 1). For example, a delivery of load 104 containing gravel, stones, and /or other materials, into an empty loading receptacle 106, may produce a specific sound, distinguishable from the sounds of other activity, objects, or events);
a phase of unloading the ore with the mining truck stationary in an unloading area (Gogolin: [0021]-[0022], [0024]-[0026], [0029], [0038], and FIG. 1-2: Such audio files may be compared with the continuous rolling audio signal sample to determine an influx (or an outflow) of load 104. Effectively, the sound-processing module 206 may process the continuous rolling audio signal sample from ambient sound signals to generate a filtered audio sample. In that way, the sound-processing module 206 may determine a load influx or a load outflow. The memory may also store a minimum decibel threshold associated with each of these materials. Moreover, inflow and outflow audio signal templates may be stored as well );
a phase of driving the mining truck empty (Gogolin: Abstract, [0021] and FIG. 1-2: audio signals from the influx of the load 104 may differ from those produced by the machine's engine, processes/operations running in the vicinity, and/or from other surrounding environmental sounds. Another factor of influx determination may be based upon the material of the load 104 (see FIG. 1). For example, a delivery of load 104 containing gravel, stones, and /or other materials, into an empty loading receptacle 106, may produce a specific sound, distinguishable from the sounds of other activity, objects, or events. ); and
at the end of the cycle, a phase of waiting for loading with the mining truck stationary ([0012]-[0013], [0028], and FIG. 1-2: The data aggregator 208 may log information upon receipt of an associated input. Input may include the filtered audio sample from the sound-processing module 206, associated with the detection of a load influx (or outflow). Segregation modules (not shown) set within the data aggregator 208, may sequentially classify each influx occurrence. Apart from logging each influx occurrence, the data aggregator 208 may also log data that corresponds to the start and stop of each influx operation. Similarly, each occurrence that corresponds to a load delivery may also be classified).
As to claim 21, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 16 further comprising the system according to claim 16, further comprising locating means mounted on or in the mining truck selected from a global positioning system (Cheng: Activity analysis of construction equipment using audio signals and support vector machines: Several remote sensing technologies such as GPS (Global Positioning System), RFID (Radio-Frequency Identification), and Ultra-Wideband (UWB) sensors can be used for location tracking of construction machines. These technologies are all based on the time-of arrival principle: ''the propagation time of a signal can be directly converted into distance if the propagation speed in known". The most popular localization system is GPS which provides location and time information anywhere on the earth if there is an unobstructed line of sight to four or more earth orbiting satellites) and an inertial navigation system.
As to claim 24, Schloss discloses a method for detecting acoustic events related to phases of a cycle of a mining truck, the method being implemented by a system installed in the mining truck and comprising the following steps:
a step of operating the system, during which step an acoustic device of the system (Schloss: FIG. 1 the one or more sensors 42 and FIG. 2 the acoustic sensors 52) captures one or more acoustic events in real time (Schloss: Abstract, [0020], [0024], [0027]-[0028], [0045], and FIG. 1-2 the acoustic sensors 52: the work machine 20 includes one or more sensors 42 that are positioned in and around the work machine 20, such as but not limited to, acoustic sensors, vision sensors, accelerometers, vibration sensors, orientation sensors and the like) and generates one or more signals indicative of each captured acoustic event (Schloss: Abstract, [0020], [0024], [0027]-[0028], [0045], and FIG. 1-2 the acoustic sensors 52 In one non-limiting example, each acoustic signal detection array 56 is configured to monitor and detect work site noise 46 (e.g., machine noise, equipment noise, traffic noise), verbal communication 50 and other such noise present in and around the work site 33); and
a step of filtering and analyzing the sent signals to determine a presence and/or absence of particular sounds and/or a frequency and duration of the particular sounds, this step being performed by a processor receiving the signals (Schloss: [0025], [0028], [0030]-[0033], [0047], [0050]-[0052], FIG. 1, and FIG. 6: Furthermore, the machine control module 44 may have one or more digital signal processors 78 (DSP) which further analyze, filter, compress or perform other such operations on the acoustic signal 58 detected by each microphone 54 and converted from an analog signal into a digital signal by the ADC 64).
Schloss does not explicitly disclose a step of sending the one or more signals to a server of the system, which step is performed by the acoustic device;
a step of filtering and analyzing the sent signals to determine a presence and/or absence of particular sounds and/or a frequency and duration of the particular sounds,
this step being performed by a processor of the server receiving the signals
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases, during which step the processor compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated,
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading and periods of driving loaded and empty to be identified from the amplitude and frequency of the recordings.
However, it has been known in the art of audio-based analysis for heave equipment to implement
a step of filtering and analyzing the sent signals to determine a presence and/or absence of particular sounds and/or a frequency and duration of the particular sounds,
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases, during which step the processor compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated,
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading identified from the amplitude and frequency of the recordings, as suggested by Cheng, which discloses
a step of filtering and analyzing the sent signals to determine a presence and/or absence of particular sounds and/or a frequency and duration of the particular sounds (Cheng: Abstract, 2.4 Gaps in knowledge: why use sound: The audio pattern generated by each individual machine is often independent of the operator and the specific way that the task is performed. Operators can perform a task in several ways. For example, imagine a hydraulic excavator is digging a trench. This operation might include a series of movements such as diggings, rotating, swinging, and loading. These tasks could be handled in various ways such as different angles, swinging to left or right, etc. A computer vision algorithm would likely need to consider all these scenarios separately, while the audio signal analysis would always yield the same result),
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases (Cheng: FIG. 5-20), during which step the processor compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated (Cheng: FIG. 5-20: Another way to visually evaluate the performance of the system is throughout the comparison charts as shown in FIG. 6, 8, 10, 12, 14, 16, 18, and 20. Tables 2-9 indicate that the performance of the proposed system for automatically recognizing activities of single machines is very promising. Generally, the proposed system can have over 80% even 85% accuracy identifying different activities. For JCB 3CX mini excavator and CAT 322C with hydraulic hammer, as shown in Tables 4 and 6, "Act 1" and "Act 2" have strongly different patterns in their STFTs, thus the identification accuracy can be over 90%. The proposed audio-based framework provides an efficient way to identify different activities for construction equipments. The machine learning algorithm needs only a few seconds of recording to obtain sufficient data to construct a model for learning audio patterns. We do not need a large database compared to neural network based machine learning method to implement the identification. Also, the machine learning model can learn the audio patterns for each activity without manual analysis of the audio recording),
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading identified from the amplitude and frequency of the recordings (Cheng: 2.3 Action recognition of construction equipment using computer vision, 2.4 Gaps in knowledge: why use sound, and FIG. 5-20: The audio pattern generated by each individual machine is often independent of the operator and the specific way that the task is performed. Operators can perform a task in several ways. For example, imagine a hydraulic excavator is digging a trench. This operation might include a series of movements such as diggings, rotating, swinging, and loading. These tasks could be handled in various ways such as different angles, swinging to left or right, etc. A computer vision algorithm would likely need to consider all these scenarios separately, while the audio signal analysis would always yield the same result ).
Therefore, in view of teachings by Schloss and Cheng, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss to implement a step of filtering and analyzing the sent signals to determine a presence and/or absence of particular sounds and/or a frequency and duration of the particular sounds,
a step of constructing a graph of sound cycles representing temporal correlations between the analyzed acoustic events and an expected chronological order of one or more associated phases, during which step the processor compares the constructed graph with one or more predefined graphs, each being indicative of the arranged and specific acoustic events and the phases with which the arranged and specific acoustic events are associated,
wherein each acoustic event is characterized by an acoustic signature allowing periods of loading and of unloading identified from the amplitude and frequency of the recordings, as suggested by Cheng. The motivation for this is to implement an audio based sensing information to monitor activities of a construction machine.
The combination of Schloss and Cheng does not explicitly disclose wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings.
However, it has been known in the art of audio-based analysis for heave equipment to implement wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings, as suggested by Gogolin, which discloses wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings (Gogolin: Abstract, [0021], [0038], and FIG. 1-2: audio signals from the influx of the load 104 may differ from those produced by the machine's engine, processes/operations running in the vicinity, and/or from other surrounding environmental sounds. Another factor of influx determination may be based upon the material of the load 104 (see FIG. 1). For example, a delivery of load 104 containing gravel, stones, and /or other materials, into an empty loading receptacle 106, may produce a specific sound, distinguishable from the sounds of other activity, objects, or events).
Therefore, in view of teachings by Schloss, Cheng, and Gogolin, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss and Cheng to implement wherein each acoustic event is characterized by an acoustic signature allowing periods of driving loaded and empty to be identified from amplitude and frequency of the recordings, as suggested by Gogolin. The motivation for this is to implement an audio based sensing information to monitor activities of a construction machine.
The combination of Schloss, Cheng, and Gogolin does not explicitly disclose a step of sending the one or more signals to a server of the system, which step is performed by the device; a step of filtering and analyzing, this step being performed by a processor of the server receiving the signals.
However, it has been known in the art of heave equipment to implement a step of sending the one or more signals to a server of the system, which step is performed by the device; a step of filtering and analyzing, this step being performed by a processor of the server receiving the signals, as suggested by Wisley, which disclose a step of sending the one or more signals to a server of the system, which step is performed by the device (Wisley: Abstract, [0042], [0046]-[0048], [0080], [0109], and FIG. 1-3: the computer system 16 may be configured to receive actual surface profile data, actual machine operating condition data and/or the actual route data and perform simulations of the operation of the at least one work machine 11. The computer system 16 may be separate from the work machine 11 and at least one surveying device 15 as illustrated (e.g. by being located in a separate housing) and they may communicate data within one another via the communication system 17. The computer system 16 may be located in a monitoring station on the worksite 14 or at a station remote to the worksite 14. For example, the computer system 16 may be located in a central server and database of the operating company of the worksite 14, the at least one surveying device 15 and/or the at least one work machine 11); and
a step of filtering and analyzing, this step being performed by a processor of the server receiving the signals (Wisley: [0058], [0076]-[0077], [0081]-[0099], and FIG. 1-3: At an analysis step 36 the monitored operating condition data may be processed in order to analyse the behaviour of the work machine 11 as it travels along the route 12. At least one type of analysis, as described below, may be carried out. The results of the analysis step 36 may be displayed to an operator on a display, such as in the form of at least one plot, gauge, map or table).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, and Wisley, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, and Gogolin to implement a step of sending the one or more signals to a server of the system, which step is performed by the device; a step of filtering and analyzing, this step being performed by a processor of the server receiving the signals, as suggested by Wisley. The motivation for this is to implement one or more processors at a server for remotely performing data analysis to determine operation conditions of a construction machine.
As to claim 27, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 24 further comprising the method according to claim 24, further comprising a step of processing a harmonic acoustic signal associated with at least one sound among an engine speed, a horn, a reversing sound and a resonance of the dump body (Schloss: [0051] and FIG. 6: the plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels, Gogolin: Abstract, [0019]-[0023], [0025]-[0026], FIG. 2, and FIG. 4), this step comprising the following steps:
a step of filtering the signal in a corresponding frequency band (Gogolin: Abstract, [0023]-[0025], [0028]-[0029], [0043]-[0044], and FIG. 1-4: a filtered audio sample is generated. By computing a score for similarity between the filtered audio sample and pre-defined inflow and outflow audio signal templates, a highest scored pattern match is provided. Upon reaching a minimum threshold, the highest scored pattern match is logged. Finally, a data event is outputted based on the highest scored pattern match on at least one of an output port and a user interface);
a step of decimating the signal (Gogolin: Abstract, [0023]-[0025], [0028]-[0029], [0043]-[0044], and FIG. 1-4: Input may include the filtered audio sample from the sound-processing module 206, associated with the detection of a load influx (or outflow). Segregation modules (not shown) set within the data aggregator 208, may sequentially classify each influx occurrence. Apart from logging each influx occurrence, the data aggregator 208 may also log data that corresponds to the start and stop of each influx operation. Similarly, each occurrence that corresponds to a load delivery may also be classified); and
a step of detecting harmonics of the resulting signal (Gogolin: Abstract, [0023]-[0025], [0028]-[0029], [0043]-[0044], and FIG. 1-4: a filtered audio sample is generated. By computing a score for similarity between the filtered audio sample and pre-defined inflow and outflow audio signal templates, a highest scored pattern match is provided. Upon reaching a minimum threshold, the highest scored pattern match is logged. Finally, a data event is outputted based on the highest scored pattern match on at least one of an output port and a user interface).
As to claim 28, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 27 further comprising the method according to claim 27, wherein the step of detecting the harmonics of the signal is performed by way of a method selected from the group consisting of comb-filtering, cepstral-analysis, spectral-autocorrelation (Gogolin: Abstract, [0023]-[0025], [0028]-[0029], [0043]-[0044], and FIG. 1-4: a filtered audio sample is generated. By computing a score for similarity between the filtered audio sample and pre-defined inflow and outflow audio signal templates, a highest scored pattern match is provided. Upon reaching a minimum threshold, the highest scored pattern match is logged. Finally, a data event is outputted based on the highest scored pattern match on at least one of an output port and a user interface), and synchronous-averaging methods.
As to claim 29, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 24 further comprising the method according to claim 24, further comprising a step of identifying expected locations of the mining truck (Wisley: [0049]-[0050], [0078], and FIG. 2: the navigation system may comprise any suitable navigation system. In particular, the at least one sensor 24 may comprise a position sensor operable to determine the position of the work machine 11 via a global navigation satellite system, such as global positioning system (GPS), or via triangulation with communication masts), wherein the identifying step comprises identifying coordinates of each expected location using data obtained by a locating means mounted on or in the mining truck (Gogolin: [0037][0047], and FIG. 1-4: In the context of fleet management, monitoring of a real-time state of the fleet includes optimizing LMT (machine 100) utilization across the mining site. A historical record of a state of the machine 100 may be applied as a basis for productivity data. As an example, a historical record may include data corresponding the number of load operation (or transfer), the location of load intake/outflow, and the duration of an intake/outflow. End users or customers may further establish a variety of the operational parameters based on these factors and Wisley: [0049]-[0050], [0078], and FIG. 2: the navigation system may comprise any suitable navigation system. In particular, the at least one sensor 24 may comprise a position sensor operable to determine the position of the work machine 11 via a global navigation satellite system, such as global positioning system (GPS), or via triangulation with communication masts).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over
Schloss et al. (Schloss - US 2019/0348057 A1) in view of Cheng et al. (Cheng – Activity Analysis of Construction Equipment Using Audio Signals and Support Vector Machines), Gogolin (Gogolin – US 2015/0020609 A1), and Wisley et al. (Wisley – US 2020/0105072 A1) and further in view of Soheili (Soheili – US 2014/0207266 A1).
As to claim 18, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 16 further comprising the system according to claim 16, wherein the acoustic signature of each acoustic event is classified into two types of sounds comprising:
a noise associated with at least one sound among an engine sound (Schloss: [0051] and FIG. 6: the plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels), a break sound and/or an ore sound; and
a harmonic noise associated with at least one sound among an engine speed (Schloss: [0051] and FIG. 6: the plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels), a horn, a reversing sound and a resonance of the dump body.
The combination of Schloss, Cheng, Gogolin, and Wisley does not explicitly disclose a white noise associated with at least one sound among an engine sound.
However, it has been known in the art of signals processing to implement a white noise associated with at least one sound among an engine sound, as suggested by Soheili, which discloses a white noise associated with at least one sound among an engine sound (Soheili: Abstract, [0056], and FIG. 1: various sound examples may be stored in the memory of the system. Audio 1 sound example may be a white noise that may mimic the engine sound of a plane).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, Wisley, and Soheili, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, Gogolin, and Wisley to implement a white noise associated with at least one sound among an engine sound, as suggested by Soheili. The motivation for this is to implement a known alternative processing for associating an engine sound using white noise.
Claims 19-20 and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over Schloss et al. (Schloss - US 2019/0348057 A1) in view of Cheng et al. (Cheng – Activity Analysis of Construction Equipment Using Audio Signals and Support Vector Machines), Gogolin (Gogolin – US 2015/0020609 A1), and Wisley et al. (Wisley – US 2020/0105072 A1) and further in view of Meier et al. (Meier – US 2021/0219493 A1).
As to claim 19, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 16 except for the claimed limitations of the system according to claim 16, wherein the acoustic recording device comprises acceleration-detecting means mounted on or in the mining truck so as to be able to detect a shock of loading and/or unloading the truck.
However, it has been known in the art of signals processing to implement wherein the acoustic recording device comprises acceleration-detecting means mounted on or in the mining truck so as to be able to detect a shock of loading and/or unloading the truck, as suggested by Meier, which discloses wherein the acoustic recording device comprises acceleration-detecting means mounted on or in the mining truck so as to be able to detect a shock of loading and/or unloading the truck (Meier: [0047]: Vibration may be measured through the use of a plurality of sensors, affixed to or otherwise coupled with the machine. The sensors are responsive to acceleration or forces. The sensors may include load cells, accelerometers, piezoelectric sensors, acoustic sensors, or any other sensor adapted to respond to force imparted on the machine such as vibration. The sensors may be responsive to vibrations in one or more modes. In one embodiment, the sensors include one or more load cells and/or accelerometers. It can be appreciated that other types of sensors, now available or later developed, may be used in lieu of, or in addition to, load cells or accelerometers [0054], [0076]-[0077], [0083], FIG. 1 the one or more of the measurement subsystems 114, and FIG. 4: Vibration V may be determined by the vibrational measurement subsystem 112 based on, for example, a measurable acceleration experienced by an accelerometer or a digital signal generated by a load cell in response to a force).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, Wisley, and Meier, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, Gogolin, and Wisley to implement wherein the acoustic recording device comprises acceleration-detecting means mounted on or in the mining truck so as to be able to detect a shock of loading and/or unloading the truck, as suggested by Meier. The motivation for this is to implement a known alternative processing to monitor activities of a construction machine.
As to claim 20, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 16 except for the claimed limitations of the system according to claim 16, further comprising at least one telematics system that monitors and records operational data of the mining truck.
However, it has been known in the art of signals processing to implement at least one telematics system that monitors and records operational data of the mining truck, as suggested by Meier, which discloses at least one telematics system (Meier: FIG. 1-2: the system 100) that monitors and records operational data of the mining truck (Meier: Abstract, [0033]-[0034], and FIG. 1-2: a computer executable program code stored in the memory 104 is executed by the processor 102 to cause the processor 102 to interact with one or more of the plurality of measurement subsystems 114. At block 202, the processor 102 acquires data from one or more of the plurality of measurement subsystems 114. At block 204, the processor 102 processes the data to determine operational state of the vehicle, At block 206, the processor 102 records the determined operational state and associated data in the memory 104 or other electronic storage device coupled with the system (not shown)).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, Wisley, and Meier, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, Gogolin, and Wisley to implement at least one telematics system that monitors and records operational data of the mining truck, as suggested by Meier. The motivation for this is to implement a known alternative processing to monitor activities of a construction machine.
As to claim 22, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 16 except for the claimed limitations of the system according to claim 16, further comprising a communication network that manages data fed to the system, the communication network incorporating at least one communication server with at least one processor that manages data corresponding to an identified mining truck.
However, it has been known in the art of signals processing to implement a communication network that manages data fed to the system, the communication network incorporating at least one communication server with at least one processor that manages data corresponding to an identified mining truck, as suggested by Meier, which discloses a communication network that manages data fed to the system, the communication network incorporating at least one communication server with at least one processor that manages data corresponding to an identified mining truck (Meier: [0033], [0094], and FIG. 1: the processor communicates the determined operational state and other information via the communications interface 106, which may include a wired and/or wireless interface such as Wi-Fi, Bluetooth, cellular, etc., to a receiver. The receiver may include a computer, a mobile device, such as a smartphone or tablet, or a remote server connected via a wired and/or wireless public and/or private communications network, such as the Internet, and the like. Wired and/or wireless Internet communication may be supported via an internal or external Internet gateway or modem).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, Wisley, and Meier, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, Gogolin, and Wisley to implement a communication network that manages data fed to the system, the communication network incorporating at least one communication server with at least one processor that manages data corresponding to an identified mining truck, as suggested by Meier. The motivation for this is to implement a known alternative processing to monitor activities of a construction machine.
As to claim 23, Schloss, Cheng, Gogolin, Wisley, and Meier disclose the limitations of claim 22 further comprising the system according to claim 22, wherein the at least one communication server is associated with one or more mining-truck managers, including one or more mining companies to which the mining truck belongs (Meier: Abstract, [0021], [0033]: the processor communicates the determined operational state and other information via the communications interface 106, which may include a wired and/or wireless interface such as Wi-Fi, Bluetooth, cellular, etc., to a receiver. The receiver may include a computer, a mobile device, such as a smartphone or tablet, or a remote server connected via a wired and/or wireless public and/or private communications network, such as the Internet, and the like. Wired and/or wireless Internet communication may be supported via an internal or external Internet gateway or modem, [0094]-[0096], [0105], FIG. 1-2, and FIG. 9-13: The simplest method of determining the source and destination of a transfer event is by detecting the time correlation of transfer events by multiple machines. The detecting the time correlation of transfer events occurs when transfer events are detected by multiple machines within the same window of time within a predetermined tolerance. The direction of the transfer may be gleaned by knowing via an initial configuration or setup the type of equipment involved in the transfer and the types of transfers associated with the equipment type).
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over
Schloss et al. (Schloss - US 2019/0348057 A1) in view of Cheng et al. (Cheng – Activity Analysis of Construction Equipment Using Audio Signals and Support Vector Machines), Gogolin (Gogolin – US 2015/0020609 A1), and Wisley et al. (Wisley – US 2020/0105072 A1) and further in view of Perrott et al. (Perrott – US 2020/0059214 A1).
As to claim 25, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 24 further comprising the method according to claim 24, further comprising a method for processing a wideband white-noise acoustic signal associated with at least one sound among an engine sound (Schloss: [0051] and FIG. 6: the plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels: In this rejection, Examiner takes Official Notice of wideband white-noise acoustic signal as a well-known signals of noises), a break sound and/or an ore sound, this step comprising the following steps:
a step of filtering the signal in a corresponding frequency band (Schloss: [0025], [0028], [0030]-[0033], [0047], [0050]-[0052], FIG. 1, and FIG. 6: Furthermore, the machine control module 44 may have one or more digital signal processors 78 (DSP) which further analyze, filter, compress or perform other such operations on the acoustic signal 58 detected by each microphone 54 and converted from an analog signal into a digital signal by the ADC 64).
The combination of Schloss, Cheng, Gogolin, and Wisley does not explicitly disclose
a decimating step, in order to limit computational load;
a step of computing a root-mean-square sound level of the filtered signal; and
a step of comparing the computed root-mean-square sound level with a predetermined threshold.
However, it has been known in the art of audio processing to implement the following steps: a decimating step, in order to limit computational load;
a step of computing a root-mean-square sound level of the filtered signal; and
a step of comparing the computed root-mean-square sound level with a predetermined threshold, as suggested by Perrott, which discloses the following steps:
a step of filtering the signal in a corresponding frequency band (Perrott: [0029]-[0030], [0033]-[0034], [0042]-[0043], [0049], [0053], FIG. 2, FIG. 6, and FIG. 9: In a further non-limiting aspect, automatic gain control in an exemplary embodiment of multipath digital signal processing signal chains can comprise a low SPL path comprising amplifier, gain stage or preamp A1 602, anti-aliasing filter (AAF1) 604, an ADC 606 such as a delta-sigma modulator (DSM1) 606, and a decimation filter (DEC1) 608, for example, as described above regarding FIG. 2),
a decimating step, in order to limit computational load (Perrott: [0029]-[0030], [0033]-[0034], [0042]-[0043], [0049], [0053], FIG. 2, FIG. 6, and FIG. 9: In a further non-limiting aspect, automatic gain control in an exemplary embodiment of multipath digital signal processing signal chains can comprise a low SPL path comprising amplifier, gain stage or preamp A1 602, anti-aliasing filter (AAF1) 604, an ADC 606 such as a delta-sigma modulator (DSM1) 606, and a decimation filter (DEC1) 608, for example, as described above regarding FIG. 2);
a step of computing a root-mean-square sound level of the filtered signal (Perrott: [0042]-[0045], [0049], [0053], [0069], [0075], FIG. 2, FIG. 6, and FIG. 9: an exemplary MUX component 432 can be configured to switch from conveying one of one or more of corrected or scaled digital audio signals to conveying a second one of the one or more of corrected or scaled digital audio signals based on switching criteria comprising or associated with amplitude measurement, absolute value of the amplitude measurement, root-mean-square power measurement of digitized data associated with the one or more digital audio signals or one or more of the digital audio signals having a characteristic measurement above a threshold); and
a step of comparing the computed root-mean-square sound level with a predetermined threshold (Perrott: [0042]-[0045], [0049], [0053], [0069], [0075], FIG. 2, FIG. 6, and FIG. 9: an exemplary MUX component 432 can be configured to switch from conveying one of one or more of corrected or scaled digital audio signals to conveying a second one of the one or more of corrected or scaled digital audio signals based on switching criteria comprising or associated with amplitude measurement, absolute value of the amplitude measurement, root-mean-square power measurement of digitized data associated with the one or more digital audio signals or one or more of the digital audio signals having a characteristic measurement above a threshold).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, Wisley, and Perrott it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, Gogolin, and Wisley to include the following steps: a decimating step, in order to limit computational load;
a step of computing a root-mean-square sound level of the filtered signal; and
a step of comparing the computed root-mean-square sound level with a predetermined threshold, as suggested by Perrott. The motivation for this is to implement a known alternative process for processing audio signals.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over
Schloss et al. (Schloss - US 2019/0348057 A1) in view of Cheng et al. (Cheng – Activity Analysis of Construction Equipment Using Audio Signals and Support Vector Machines), Gogolin (Gogolin – US 2015/0020609 A1), and Wisley et al. (Wisley – US 2020/0105072 A1) and further in view of Korakin et al. (Korakin – US 2014/0035750 A1).
As to claim 26, Schloss, Cheng, Gogolin, and Wisley disclose the limitations of claim 24 further comprising the method according to claim 24, further comprising a step of processing a wideband white-noise acoustic signal associated with at least one sound among an engine sound (Schloss: [0051] and FIG. 6: the plurality of acoustic sensors 52 are activated to monitor and collect engine noise generated during various operational states of the work machine 20. For example, engine noise may be measured and analyzed at idle speed as well as at several different engine speeds run at pre-determined RPM levels. In this rejection, Examiner takes Official Notice of wideband white-noise acoustic signal as a well-known signals of noises), a break sound and/or an ore sound.
The combination of Schloss, Cheng, Gogolin, and Wisley does not explicitly disclose this step comprising the following steps: a step of computing a frequency spectrum in a sliding window of the time-domain signal; and a step of deducing acoustic power via integration in a desired frequency band.
However, it has been known in the art of audio processing to implement the following steps: a step of computing a frequency spectrum in a sliding window of the time-domain signal; and a step of deducing acoustic power via integration in a desired frequency band, as suggested by Korakin, which discloses the following steps: a step of computing a frequency spectrum in a sliding window of the time-domain signal (Korakin: [0024]-[0025], [0034]-[0035], [0066]-[0067], [0091]-[0095], and FIG. 11-12: Receiving sound samples using microphone or file as a source. The source is being segmented using a configurable time window, for example 1 second. Using a spectrogram operation we convert the signal into three dimensional function S which represent the signal power at a certain time frequency point (see, for example graph 550 of FIG. 11)); and a step of deducing acoustic power via integration in a desired frequency band (Korakin: [0024]-[0025], [0034]-[0035], [0066]-[0067], [0091]-[0095], and FIG. 11-12: the method may include calculating, a for each point of time out of multiple points in time of a time window, a sum of amplitudes of spectral components of the detection signals within a frequency window thereby providing multiple sums associated with the multiple points in time; and processing the multiple sums to search for a signature that is characteristic of a hazardous animal ).
Therefore, in view of teachings by Schloss, Cheng, Gogolin, Wisley, and Korakin it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss, Cheng, Gogolin, and Wisley to include the following steps: a step of computing a frequency spectrum in a sliding window of the time-domain signal; and a step of deducing acoustic power via integration in a desired frequency band, as suggested by Korakin. The motivation for this is to implement a known alternative process for processing audio signals in order to determine a specific sound.
Claim 30 is rejected under 35 U.S.C. 103 as being unpatentable over Schloss et al. (Schloss - US 2019/0348057 A1) in view of Cheng et al. (Cheng – Activity Analysis of Construction Equipment Using Audio Signals and Support Vector Machines).
As to claim 30, Schloss discloses a mining truck comprising:
a dump body pivotally mounted on a frame (Schloss: FIG. 1 the work machine 20);
a cab in which an operator of the mining truck sits (Schloss: [0017], [0020], [0022], [0025], [0032]-[0034], and FIG. 1 the operator compartment 26: moreover, an embodiment of the machine 20 includes a frame 24 which provides support to the engine 22, an operator compartment 26 and other such components of the work machine 20. Furthermore, the operator compartment 26 defines a fully enclosed area, or in some cases semi-enclosed, for an operator of the machine 20 to sit and/or stand in while operating the machine);
an engine associated with the frame (Schloss: [0017]-[0018], [0021], [0026], [0030]-[0031], [0051], and FIG. 1 the engine 22: The work machine 20 may include an engine 22 configured to supply power to the machine, such as but not limited to, a diesel engine, a gasoline internal combustion engine, a natural gas engine, an electric motor, and other known power generating sources or combinations thereof. Moreover, an embodiment of the machine 20 includes a frame 24 which provides support to the engine 22, an operator compartment 26 and other such components of the work machine 20).
Schloss does not explicitly disclose a module for executing a method allowing acoustic events denoting phases of a cycle of the mining truck to be detected, the executing module comprising an analyzing application for analyzing one or more signals indicative of the acoustic events in order to determine whether the mining truck is in a course of a mining cycle and, when an acoustic event represented by these signals is present, to determine a current phase of the mining cycle being executed by the mining truck.
However, it has been known in the art of audio-based analysis for heave equipment to implement a module for executing a method allowing acoustic events denoting phases of a cycle of the mining truck to be detected, the executing module comprising an analyzing application for analyzing one or more signals indicative of the acoustic events in order to determine whether the mining truck is in a course of a mining cycle and, when an acoustic event represented by these signals is present, to determine a current phase of the mining cycle being executed by the mining truck, as suggested by Cheng, which discloses a module for executing a method allowing acoustic events denoting phases of a cycle of the mining truck to be detected (Cheng: Abstract, 2.4 Gaps in knowledge: why use sound: The audio pattern generated by each individual machine is often independent of the operator and the specific way that the task is performed. Operators can perform a task in several ways. For example, imagine a hydraulic excavator is digging a trench. This operation might include a series of movements such as diggings, rotating, swinging, and loading. These tasks could be handled in various ways such as different angles, swinging to left or right, etc. A computer vision algorithm would likely need to consider all these scenarios separately, while the audio signal analysis would always yield the same result),
the executing module comprising an analyzing application for analyzing one or more signals indicative of the acoustic events in order to determine whether the mining truck is in a course of a mining cycle (Cheng: FIG. 5-20: Another way to visually evaluate the performance of the system is throughout the comparison charts as shown in FIG. 6, 8, 10, 12, 14, 16, 18, and 20. Tables 2-9 indicate that the performance of the proposed system for automatically recognizing activities of single machines is very promising. Generally, the proposed system can have over 80% even 85% accuracy identifying different activities. For JCB 3CX mini excavator and CAT 322C with hydraulic hammer, as shown in Tables 4 and 6, "Act 1" and "Act 2" have strongly different patterns in their STFTs, thus the identification accuracy can be over 90%. The proposed audio-based framework provides an efficient way to identify different activities for construction equipments. The machine learning algorithm needs only a few seconds of recording to obtain sufficient data to construct a model for learning audio patterns. We do not need a large database compared to neural network based machine learning method to implement the identification. Also, the machine learning model can learn the audio patterns for each activity without manual analysis of the audio recording), and
when an acoustic event represented by these signals is present, to determine a current phase of the mining cycle being executed by the mining truck (Cheng: 2.3 Action recognition of construction equipment using computer vision, 2.4 Gaps in knowledge: why use sound, and FIG. 5-20: The audio pattern generated by each individual machine is often independent of the operator and the specific way that the task is performed. Operators can perform a task in several ways. For example, imagine a hydraulic excavator is digging a trench. This operation might include a series of movements such as diggings, rotating, swinging, and loading. These tasks could be handled in various ways such as different angles, swinging to left or right, etc. A computer vision algorithm would likely need to consider all these scenarios separately, while the audio signal analysis would always yield the same result ).
Therefore, in view of teachings by Schloss and Cheng, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to implement in the construction machine of Schloss to implement a module for executing a method allowing acoustic events denoting phases of a cycle of the mining truck to be detected, the executing module comprising an analyzing application for analyzing one or more signals indicative of the acoustic events in order to determine whether the mining truck is in a course of a mining cycle and, when an acoustic event represented by these signals is present, to determine a current phase of the mining cycle being executed by the mining truck, as suggested by Cheng. The motivation for this is to implement an audio based sensing information to monitor activities of a construction machine.
Citation of Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Petrany et al., US 12,252,870 B2, discloses ground engaging tool wear and loss detection system and method.
Kurosawa, US 2022/0018097 A1, discloses information processing apparatus, information processing method, and recording medium, and work machine.
Izumikawa et al., US 2019/0241124 A1, discloses construction machine safety management system, management apparatus.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUANG PHAM whose telephone number is (571)-270-3668. The examiner can normally be reached 09:00 AM - 05:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, QUAN-ZHEN WANG can be reached at (571)-272-3114. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUANG PHAM/Primary Examiner, Art Unit 2685