DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20, as originally filed, are currently pending and have been considered below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Clifton et al., U.S. Publication No. 2020/0348681, hereinafter, “Clifton”, and further in view of Mahan et al., U.S. Publication No. 2021/0292086, hereinafter, “Mahan”.
As per claim 1, Clifton discloses a system for verifying refuse collection, the system comprising:
a camera system coupled to a refuse vehicle and configured to detect an object proximate to the refuse vehicle (Clifton, ¶0003, An imaging device is in communication with the processing unit and is positioned on the refuse vehicle to have a field of view extending outwardly away from the refuse vehicle. The processing unit controls the imaging device to capture an image upon receiving an indication, from the sensor, that an indicator is present within the field of view. In some embodiments, the indicator is the presence of a positive object, like a waste container. In other embodiments, the indicator is the omission of an object (e.g., no container is detected) within the field of view; Clifton, ¶0018, the refuse vehicle 10 includes an imaging device 32 that is used to capture images or videos of vehicle surroundings and environment as the refuse vehicle 10 operates ... The imaging device 32 can be located on the vehicle body 18, lifting system 26, or some other location on the vehicle so that the imaging device field of view 34 extends at least partially outwardly away from the refuse vehicle 10; Clifton, ¶0019, the area sensor 36 can actively monitor an area sensor field of view 38 for the presence of an indicator 40, like a waste container. If the area sensor 36 determines that an indicator 40 is present within the area sensor field of view 38, the area sensor 36 can provide an indication to the processing unit 30); and
one or more memory devices storing instructions thereon that, when executed by one or more processors (Clifton, ¶0020, The memory 48 can be local or remote memory in communication with the processing unit 30. For example, cloud-based or network-based memory devices can be used, as well as on-board random access memory (RAM)), cause the one or more processors to:
receive image data from the camera system (Clifton, ¶0003, An imaging device is in communication with the processing unit and is positioned on the refuse vehicle to have a field of view extending outwardly away from the refuse vehicle. The processing unit controls the imaging device to capture media; Clifton, ¶0018, the refuse vehicle 10 includes an imaging device 32 that is used to capture images or videos of vehicle surroundings and environment as the refuse vehicle 10 operates ... The imaging device 32 can be located on the vehicle body 18, lifting system 26, or some other location on the vehicle so that the imaging device field of view 34 extends at least partially outwardly away from the refuse vehicle 10);
determine, based on stored data, whether the object is expected to be present for collection at a location (Clifton, ¶0023, the processing unit 30 actively monitors the current location of the refuse vehicle 10 using the GPS 44. The address of each customer can be stored within the memory 48, and the processing unit 30 can be actively comparing the current location of the refuse vehicle 10 to the stored customer addresses within the memory 48. Whenever the current location of the refuse vehicle 10 matches a stored address of a customer, the processing unit 30 can issue a command to the area sensor 36 to perform step 102, and scan the area sensor field of view 38 for an indicator 40); and
based on determining that the object is expected to be present for collection at the location, instruct the camera system to store an image of the location on the one or more memory devices (Clifton, ¶0005, The processing unit controls the imaging device to capture media … and then directs a memory to store the captured media with a current refuse vehicle location generated by the global positioning system and a timestamp generated by the clock; Clifton, ¶0014, Refuse vehicles are used to pick up refuse at multiple locations during a single trip. Events may occur along the trip that could be recorded to aid in future decision-making processes. For example, customers along a garbage or recycling route may forget to put out waste containers prior to the refuse vehicle traversing a scheduled waste pick-up route. Documenting and imaging the missing waste container, along with the timing and location where the can was missing, would provide evidence that proper protocols were followed; Clifton, ¶0020, The clock 46 can provide a time stamp for when the imaging device 32 operated (i.e., captured and/or recorded media), which can then be assigned to the particular piece of recorded media (i.e., image(s) or video). Data from the GPS 44 and clock 46 can be provided to the processing unit 30, which can then group the captured media and informational data together and store each piece of information as an event on a memory 48; Clifton, ¶0024, At decision block 104, the area sensor 36 determines whether or not an indicator 40 is present within the area sensor field of view 38. The indicator 40 can be both positive or negative items. For example, and as shown in FIG. 4, the indicator 40 could be one or more waste containers. In some embodiments, the indicator 40 could be the presence of non-compliant waste containers; Clifton, ¶0025, In other embodiments, and as shown in FIG. 5, the indicator 40 is a negative object (i.e., lack of something) within the area sensor field of view 38. For example, the indicator 40 can be the lack of a waste container within the area sensor field of view. When the refuse vehicle 10 arrives at an address associated with a customer, the area sensor 36 can continue scanning the area sensor field of view 38 for the presence of a waste container; Clifton, ¶0027, the GPS 44 and clock 46 can supply an exact location and a time stamp for when the area sensor 36 detected the presence of the indicator 40 within the area sensor field of view 38, and the time in which the event occurred. Each of these pieces of information can then be associated with the captured media and stored within the memory 48, where they may be accessed at a later time).
Clifton further discloses (Clifton, ¶0021, the transmitter 52 issues a signal containing the time, address, and media associated with the event each time the imaging device 32 operates to capture and record media. The communications module 50 can further include a receiver 54 to receive instructions from an external source, like the fleet command center 56. In some examples, the imaging device 32 captures media and sends the media, through the transmitter 52, to an external computing device having an image analyzer (not shown). The image analyzer can determine whether or not an indicator 40 is present within the transmitted media) but does not explicitly disclose the following limitation as further recited however Mahan discloses
transmit the image data to an object detection system (Mahan, ¶0003, a controller configured to receive first data from the at least one sensor, input the first data to a single-stage object detector, identify, based on an output of the single-stage object detector, the refuse can, and initiate a control action to move the actuator assembly and the refuse collection vehicle to engage the refuse can; Mahan, ¶0017, The refuse can detection systems may include a controller configured to receive and process data from a plurality of cameras and/or sensors coupled to a refuse vehicle ... The plurality of cameras and/or sensors (e.g., LIDAR, radar, etc.) and the controller may be disposed in any suitable location on the refuse vehicle. The controller may process data from the cameras and/or sensors to detect the presence of refuse cans; Mahan, ¶0045, referring to FIG. 4, memory 406 is shown to include an object detector 420. Object detector 420 may generally receive and process data from image/object sensors 430 to detect objects (e.g., refuse cans); Mahan, ¶0067, At 712, a determination is made based on the whether or not a refuse can (e.g., or multiple refuse cans) are detected based on the data).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Mahan and Clifton because they are in the same field of endeavor. One skilled in the art would have been motivated to substitute the object detection system as taught by Mahan for the image analyzer as taught by Clifton as an alternate means to determine if expected objects or refuse containers are present in the captured images (Mahan, ¶0072).
As per claim 2, Clifton and Mahan disclose the system of claim 1, wherein the object is a refuse can (Clifton, ¶0003, the indicator is the presence of a positive object, like a waste container. In other embodiments, the indicator is the omission of an object (e.g., no container is detected) within the field of view).
As per claim 3, Clifton and Mahan disclose the system of claim 1, wherein the instructions further cause the one or more processors to:
based on an output from the object detection system, determine whether the object is present at the location (Clifton, ¶0021, the transmitter 52 issues a signal containing the time, address, and media associated with the event each time the imaging device 32 operates to capture and record media. The communications module 50 can further include a receiver 54 to receive instructions from an external source, like the fleet command center 56. In some examples, the imaging device 32 captures media and sends the media, through the transmitter 52, to an external computing device having an image analyzer (not shown). The image analyzer can determine whether or not an indicator 40 is present within the transmitted media; Mahan, ¶0003, a controller configured to receive first data from the at least one sensor, input the first data to a single-stage object detector, identify, based on an output of the single-stage object detector, the refuse can, and initiate a control action to move the actuator assembly and the refuse collection vehicle to engage the refuse can); and
determine whether the object is collected (Mahan, ¶0057, Control module 424 determine and/or implement control actions based on detected objects (e.g., from object detector 420) and/or user inputs (e.g., from user interface 432). In some embodiments, control module 424 may implement any number of automated control actions based on detected objects such as refuse cans and/or human beings. In a first example, control module 424 may implement automated collection of a refuse can, based on detection of the refuse can. In this example, once a refuse can is detected, a location of the refuse can may be determined using any number of known methods. Based on the determined location of the target refuse can, control module 424 may determine a trajectory for refuse vehicle 10 and/or actuator assembly 436 in order to engage the refuse can).
As per claim 4, Clifton and Mahan disclose the system of claim 1, wherein the one or more processors instruct the camera system to store the image of the location based on determining at least one of: the object is present at the location, the object is expected to be present for collection at the location, and the object is not collected; the object is not present at the location, the object is expected to be present for collection at the location, and the object is not collected; and the object is present at the location and the object is not expected to be present at the location (Clifton, ¶0028, The method 100 can include further steps of alerting customers when an indicator 40 was identified at the address associated with their account. For example, the processing unit 30 could issue a notification (e.g., an email or a text message) explaining that no waste container was found on their premises that day. In some embodiments, the notification may provide an option to the customer to arrange for supplemental pick-up service, which could be offered at an additional cost. The media could also be used to provide evidence in case a customer calls to dispute whether or not the waste container was properly accessible on the day of the waste removal route).
As per claim 5, Clifton and Mahan disclose the system of claim 1, wherein the instructions further cause the one or more processors to store data relating to collection of the object, the data comprising at least in part a determination of whether or not the object is collected (Clifton, ¶0027, the GPS 44 and clock 46 can supply an exact location and a time stamp for when the area sensor 36 detected the presence of the indicator 40 within the area sensor field of view 38, and the time in which the event occurred. Each of these pieces of information can then be associated with the captured media and stored within the memory 48, where they may be accessed at a later time; Clifton, ¶0028, The method 100 can include further steps of alerting customers when an indicator 40 was identified at the address associated with their account. For example, the processing unit 30 could issue a notification (e.g., an email or a text message) explaining that no waste container was found on their premises that day. In some embodiments, the notification may provide an option to the customer to arrange for supplemental pick-up service, which could be offered at an additional cost. The media could also be used to provide evidence in case a customer calls to dispute whether or not the waste container was properly accessible on the day of the waste removal route).
As per claim 6, Clifton and Mahan disclose the system of claim 5, wherein the instructions further cause the one or more processors to, based on a determination that the object is not collected, provide a driver of the refuse vehicle with one or more options relating to the data (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications).
As per claim 7, Clifton and Mahan disclose the system of claim 6, wherein the one or more options comprise a menu containing additional information, an option for entering information, or a training module (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications).
As per claim 8, Clifton and Mahan disclose the system of claim 1, wherein the instructions further cause the one or more processors to transmit a notification to a driver of the refuse vehicle (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications. In some embodiments, the user interface generated by UI manager 422 may include a notification when a human being is detected within a danger zone. This may alert an operator to an unsafe condition and/or may indicate to the operator why automated refuse can collection cannot be implemented).
As per claim 9, Clifton discloses a method for verifying refuse collection, the method comprising:
receiving image data from a camera system coupled to a refuse vehicle (Clifton, ¶0003, An imaging device is in communication with the processing unit and is positioned on the refuse vehicle to have a field of view extending outwardly away from the refuse vehicle. The processing unit controls the imaging device to capture an image upon receiving an indication, from the sensor, that an indicator is present within the field of view. In some embodiments, the indicator is the presence of a positive object, like a waste container. In other embodiments, the indicator is the omission of an object (e.g., no container is detected) within the field of view; Clifton, ¶0018, the refuse vehicle 10 includes an imaging device 32 that is used to capture images or videos of vehicle surroundings and environment as the refuse vehicle 10 operates ... The imaging device 32 can be located on the vehicle body 18, lifting system 26, or some other location on the vehicle so that the imaging device field of view 34 extends at least partially outwardly away from the refuse vehicle 10; Clifton, ¶0019, the area sensor 36 can actively monitor an area sensor field of view 38 for the presence of an indicator 40, like a waste container. If the area sensor 36 determines that an indicator 40 is present within the area sensor field of view 38, the area sensor 36 can provide an indication to the processing unit 30);
processing the image data (Clifton, ¶0021, the transmitter 52 issues a signal containing the time, address, and media associated with the event each time the imaging device 32 operates to capture and record media. The communications module 50 can further include a receiver 54 to receive instructions from an external source, like the fleet command center 56. In some examples, the imaging device 32 captures media and sends the media, through the transmitter 52, to an external computing device having an image analyzer (not shown). The image analyzer can determine whether or not an indicator 40 is present within the transmitted media); and
determining one or more of: based on stored data, whether an object is expected to be present for collection at a location (Clifton, ¶0023, the processing unit 30 actively monitors the current location of the refuse vehicle 10 using the GPS 44. The address of each customer can be stored within the memory 48, and the processing unit 30 can be actively comparing the current location of the refuse vehicle 10 to the stored customer addresses within the memory 48. Whenever the current location of the refuse vehicle 10 matches a stored address of a customer, the processing unit 30 can issue a command to the area sensor 36 to perform step 102, and scan the area sensor field of view 38 for an indicator 40); and
based on determining whether the object is expected to be present for collection at the location, instructing the camera system to store an image of the location on a memory (Clifton, ¶0005, The processing unit controls the imaging device to capture media … and then directs a memory to store the captured media with a current refuse vehicle location generated by the global positioning system and a timestamp generated by the clock; Clifton, ¶0014, Refuse vehicles are used to pick up refuse at multiple locations during a single trip. Events may occur along the trip that could be recorded to aid in future decision-making processes. For example, customers along a garbage or recycling route may forget to put out waste containers prior to the refuse vehicle traversing a scheduled waste pick-up route. Documenting and imaging the missing waste container, along with the timing and location where the can was missing, would provide evidence that proper protocols were followed; Clifton, ¶0020, The clock 46 can provide a time stamp for when the imaging device 32 operated (i.e., captured and/or recorded media), which can then be assigned to the particular piece of recorded media (i.e., image(s) or video). Data from the GPS 44 and clock 46 can be provided to the processing unit 30, which can then group the captured media and informational data together and store each piece of information as an event on a memory 48; Clifton, ¶0024, At decision block 104, the area sensor 36 determines whether or not an indicator 40 is present within the area sensor field of view 38. The indicator 40 can be both positive or negative items. For example, and as shown in FIG. 4, the indicator 40 could be one or more waste containers. In some embodiments, the indicator 40 could be the presence of non-compliant waste containers; Clifton, ¶0025, In other embodiments, and as shown in FIG. 5, the indicator 40 is a negative object (i.e., lack of something) within the area sensor field of view 38. For example, the indicator 40 can be the lack of a waste container within the area sensor field of view. When the refuse vehicle 10 arrives at an address associated with a customer, the area sensor 36 can continue scanning the area sensor field of view 38 for the presence of a waste container; Clifton, ¶0027, the GPS 44 and clock 46 can supply an exact location and a time stamp for when the area sensor 36 detected the presence of the indicator 40 within the area sensor field of view 38, and the time in which the event occurred. Each of these pieces of information can then be associated with the captured media and stored within the memory 48, where they may be accessed at a later time).
Clifton further discloses (Clifton, ¶0021, the transmitter 52 issues a signal containing the time, address, and media associated with the event each time the imaging device 32 operates to capture and record media. The communications module 50 can further include a receiver 54 to receive instructions from an external source, like the fleet command center 56. In some examples, the imaging device 32 captures media and sends the media, through the transmitter 52, to an external computing device having an image analyzer (not shown). The image analyzer can determine whether or not an indicator 40 is present within the transmitted media) but does not explicitly disclose the following limitation as further recited however Mahan discloses
inputting the image data into an object detection system (Mahan, ¶0003, a controller configured to receive first data from the at least one sensor, input the first data to a single-stage object detector, identify, based on an output of the single-stage object detector, the refuse can, and initiate a control action to move the actuator assembly and the refuse collection vehicle to engage the refuse can; Mahan, ¶0017, The refuse can detection systems may include a controller configured to receive and process data from a plurality of cameras and/or sensors coupled to a refuse vehicle ... The plurality of cameras and/or sensors (e.g., LIDAR, radar, etc.) and the controller may be disposed in any suitable location on the refuse vehicle. The controller may process data from the cameras and/or sensors to detect the presence of refuse cans; Mahan, ¶0045, referring to FIG. 4, memory 406 is shown to include an object detector 420. Object detector 420 may generally receive and process data from image/object sensors 430 to detect objects (e.g., refuse cans); Mahan, ¶0067, At 712, a determination is made based on the whether or not a refuse can (e.g., or multiple refuse cans) are detected based on the data).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Mahan and Clifton because they are in the same field of endeavor. One skilled in the art would have been motivated to substitute the object detection system as taught by Mahan for the image analyzer as taught by Clifton as an alternate means to determine if expected objects or refuse containers are present in the captured images (Mahan, ¶0072).
As per claim 10, Clifton and Mahan disclose the method of claim 9, wherein the object is a refuse can (Clifton, ¶0003, the indicator is the presence of a positive object, like a waste container. In other embodiments, the indicator is the omission of an object (e.g., no container is detected) within the field of view).
As per claim 11, Clifton and Mahan disclose the method of claim 9, wherein instructing the camera system to store the image of the location is based on determining at least one of: the object is present at the location, the object is expected to be present for collection at the location, and the object is not collected; the object is not present at the location, the object is expected to be present for collection at the location, and the object is not collected; and the object is present at the location and the object is not expected to be present at the location for collection (Clifton, ¶0028, The method 100 can include further steps of alerting customers when an indicator 40 was identified at the address associated with their account. For example, the processing unit 30 could issue a notification (e.g., an email or a text message) explaining that no waste container was found on their premises that day. In some embodiments, the notification may provide an option to the customer to arrange for supplemental pick-up service, which could be offered at an additional cost. The media could also be used to provide evidence in case a customer calls to dispute whether or not the waste container was properly accessible on the day of the waste removal route).
As per claim 12, Clifton and Mahan disclose the method of claim 9, further comprising storing data relating to collection of the object, the data comprising at least in part a determination of whether or not the object is collected (Clifton, ¶0027, the GPS 44 and clock 46 can supply an exact location and a time stamp for when the area sensor 36 detected the presence of the indicator 40 within the area sensor field of view 38, and the time in which the event occurred. Each of these pieces of information can then be associated with the captured media and stored within the memory 48, where they may be accessed at a later time; Clifton, ¶0028, The method 100 can include further steps of alerting customers when an indicator 40 was identified at the address associated with their account. For example, the processing unit 30 could issue a notification (e.g., an email or a text message) explaining that no waste container was found on their premises that day. In some embodiments, the notification may provide an option to the customer to arrange for supplemental pick-up service, which could be offered at an additional cost. The media could also be used to provide evidence in case a customer calls to dispute whether or not the waste container was properly accessible on the day of the waste removal route).
As per claim 13, Clifton and Mahan disclose the method of claim 12, further comprising, based on a determination that the object is not collected, providing a driver of the refuse vehicle with one or more options relating to the data (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications).
As per claim 14, Clifton and Mahan disclose the method of claim 13, wherein providing the driver of the refuse vehicle with one or more options relating to the data comprises providing the driver with a menu containing additional information, an option for entering information, or a training module (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications).
As per claim 15, Clifton and Mahan disclose the method of claim 9, further comprising transmitting a notification to a driver of the refuse vehicle (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications. In some embodiments, the user interface generated by UI manager 422 may include a notification when a human being is detected within a danger zone. This may alert an operator to an unsafe condition and/or may indicate to the operator why automated refuse can collection cannot be implemented).
As per claim 16, Clifton discloses a refuse vehicle, comprising:
a camera system configured to detect an object proximate to the refuse vehicle (Clifton, ¶0003, An imaging device is in communication with the processing unit and is positioned on the refuse vehicle to have a field of view extending outwardly away from the refuse vehicle. The processing unit controls the imaging device to capture an image upon receiving an indication, from the sensor, that an indicator is present within the field of view. In some embodiments, the indicator is the presence of a positive object, like a waste container. In other embodiments, the indicator is the omission of an object (e.g., no container is detected) within the field of view; Clifton, ¶0018, the refuse vehicle 10 includes an imaging device 32 that is used to capture images or videos of vehicle surroundings and environment as the refuse vehicle 10 operates ... The imaging device 32 can be located on the vehicle body 18, lifting system 26, or some other location on the vehicle so that the imaging device field of view 34 extends at least partially outwardly away from the refuse vehicle 10; Clifton, ¶0019, the area sensor 36 can actively monitor an area sensor field of view 38 for the presence of an indicator 40, like a waste container. If the area sensor 36 determines that an indicator 40 is present within the area sensor field of view 38, the area sensor 36 can provide an indication to the processing unit 30); and
one or more memory devices storing instructions thereon that, when executed by one or more processors (Clifton, ¶0020, The memory 48 can be local or remote memory in communication with the processing unit 30. For example, cloud-based or network-based memory devices can be used, as well as on-board random access memory (RAM)), cause the one or more processors to:
receive image data from the camera system (Clifton, ¶0003, An imaging device is in communication with the processing unit and is positioned on the refuse vehicle to have a field of view extending outwardly away from the refuse vehicle. The processing unit controls the imaging device to capture media; Clifton, ¶0018, the refuse vehicle 10 includes an imaging device 32 that is used to capture images or videos of vehicle surroundings and environment as the refuse vehicle 10 operates ... The imaging device 32 can be located on the vehicle body 18, lifting system 26, or some other location on the vehicle so that the imaging device field of view 34 extends at least partially outwardly away from the refuse vehicle 10);
determine, based on stored data, whether the object is expected to be present for collection at a location (Clifton, ¶0023, the processing unit 30 actively monitors the current location of the refuse vehicle 10 using the GPS 44. The address of each customer can be stored within the memory 48, and the processing unit 30 can be actively comparing the current location of the refuse vehicle 10 to the stored customer addresses within the memory 48. Whenever the current location of the refuse vehicle 10 matches a stored address of a customer, the processing unit 30 can issue a command to the area sensor 36 to perform step 102, and scan the area sensor field of view 38 for an indicator 40); and
based on determining that the object is expected to be present for collection at the location, instruct the camera system to store an image of the location on the one or more memory devices (Clifton, ¶0005, The processing unit controls the imaging device to capture media … and then directs a memory to store the captured media with a current refuse vehicle location generated by the global positioning system and a timestamp generated by the clock; Clifton, ¶0014, Refuse vehicles are used to pick up refuse at multiple locations during a single trip. Events may occur along the trip that could be recorded to aid in future decision-making processes. For example, customers along a garbage or recycling route may forget to put out waste containers prior to the refuse vehicle traversing a scheduled waste pick-up route. Documenting and imaging the missing waste container, along with the timing and location where the can was missing, would provide evidence that proper protocols were followed; Clifton, ¶0020, The clock 46 can provide a time stamp for when the imaging device 32 operated (i.e., captured and/or recorded media), which can then be assigned to the particular piece of recorded media (i.e., image(s) or video). Data from the GPS 44 and clock 46 can be provided to the processing unit 30, which can then group the captured media and informational data together and store each piece of information as an event on a memory 48; Clifton, ¶0024, At decision block 104, the area sensor 36 determines whether or not an indicator 40 is present within the area sensor field of view 38. The indicator 40 can be both positive or negative items. For example, and as shown in FIG. 4, the indicator 40 could be one or more waste containers. In some embodiments, the indicator 40 could be the presence of non-compliant waste containers; Clifton, ¶0025, In other embodiments, and as shown in FIG. 5, the indicator 40 is a negative object (i.e., lack of something) within the area sensor field of view 38. For example, the indicator 40 can be the lack of a waste container within the area sensor field of view. When the refuse vehicle 10 arrives at an address associated with a customer, the area sensor 36 can continue scanning the area sensor field of view 38 for the presence of a waste container; Clifton, ¶0027, the GPS 44 and clock 46 can supply an exact location and a time stamp for when the area sensor 36 detected the presence of the indicator 40 within the area sensor field of view 38, and the time in which the event occurred. Each of these pieces of information can then be associated with the captured media and stored within the memory 48, where they may be accessed at a later time).
Clifton further discloses (Clifton, ¶0021, the transmitter 52 issues a signal containing the time, address, and media associated with the event each time the imaging device 32 operates to capture and record media. The communications module 50 can further include a receiver 54 to receive instructions from an external source, like the fleet command center 56. In some examples, the imaging device 32 captures media and sends the media, through the transmitter 52, to an external computing device having an image analyzer (not shown). The image analyzer can determine whether or not an indicator 40 is present within the transmitted media) but does not explicitly disclose the following limitation as further recited however Mahan discloses
transmit the image data to an object detection system (Mahan, ¶0003, a controller configured to receive first data from the at least one sensor, input the first data to a single-stage object detector, identify, based on an output of the single-stage object detector, the refuse can, and initiate a control action to move the actuator assembly and the refuse collection vehicle to engage the refuse can; Mahan, ¶0017, The refuse can detection systems may include a controller configured to receive and process data from a plurality of cameras and/or sensors coupled to a refuse vehicle ... The plurality of cameras and/or sensors (e.g., LIDAR, radar, etc.) and the controller may be disposed in any suitable location on the refuse vehicle. The controller may process data from the cameras and/or sensors to detect the presence of refuse cans; Mahan, ¶0045, referring to FIG. 4, memory 406 is shown to include an object detector 420. Object detector 420 may generally receive and process data from image/object sensors 430 to detect objects (e.g., refuse cans); Mahan, ¶0067, At 712, a determination is made based on the whether or not a refuse can (e.g., or multiple refuse cans) are detected based on the data).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Mahan and Clifton because they are in the same field of endeavor. One skilled in the art would have been motivated to substitute the object detection system as taught by Mahan for the image analyzer as taught by Clifton as an alternate means to determine if expected objects or refuse containers are present in the captured images (Mahan, ¶0072).
As per claim 17, Clifton and Mahan disclose the refuse vehicle of claim 16, wherein the instructions further cause the one or more processors to:
based on an output from the object detection system, determine whether the object is present at the location (Clifton, ¶0021, the transmitter 52 issues a signal containing the time, address, and media associated with the event each time the imaging device 32 operates to capture and record media. The communications module 50 can further include a receiver 54 to receive instructions from an external source, like the fleet command center 56. In some examples, the imaging device 32 captures media and sends the media, through the transmitter 52, to an external computing device having an image analyzer (not shown). The image analyzer can determine whether or not an indicator 40 is present within the transmitted media; Mahan, ¶0003, a controller configured to receive first data from the at least one sensor, input the first data to a single-stage object detector, identify, based on an output of the single-stage object detector, the refuse can, and initiate a control action to move the actuator assembly and the refuse collection vehicle to engage the refuse can); and
determine whether the object is collected (Mahan, ¶0057, Control module 424 determine and/or implement control actions based on detected objects (e.g., from object detector 420) and/or user inputs (e.g., from user interface 432). In some embodiments, control module 424 may implement any number of automated control actions based on detected objects such as refuse cans and/or human beings. In a first example, control module 424 may implement automated collection of a refuse can, based on detection of the refuse can. In this example, once a refuse can is detected, a location of the refuse can may be determined using any number of known methods. Based on the determined location of the target refuse can, control module 424 may determine a trajectory for refuse vehicle 10 and/or actuator assembly 436 in order to engage the refuse can).
As per claim 18, Clifton and Mahan disclose the refuse vehicle of claim 16, wherein the one or more processors instruct the camera system to store the image of the location based on determining at least one of: the object is present at the location, the object is expected to be present for collection at the location, and the object is not collected; the object is not present at the location, the object is expected to be present for collection at the location, and the object is not collected; and the object is present at the location and the object is not expected to be present at the location (Clifton, ¶0028, The method 100 can include further steps of alerting customers when an indicator 40 was identified at the address associated with their account. For example, the processing unit 30 could issue a notification (e.g., an email or a text message) explaining that no waste container was found on their premises that day. In some embodiments, the notification may provide an option to the customer to arrange for supplemental pick-up service, which could be offered at an additional cost. The media could also be used to provide evidence in case a customer calls to dispute whether or not the waste container was properly accessible on the day of the waste removal route).
As per claim 19, Clifton and Mahan disclose the refuse vehicle of claim 16, wherein the instructions further cause the one or more processors to store data relating to collection of the object, the data comprising at least in part a determination of whether or not the object is collected (Clifton, ¶0027, the GPS 44 and clock 46 can supply an exact location and a time stamp for when the area sensor 36 detected the presence of the indicator 40 within the area sensor field of view 38, and the time in which the event occurred. Each of these pieces of information can then be associated with the captured media and stored within the memory 48, where they may be accessed at a later time; Clifton, ¶0028, The method 100 can include further steps of alerting customers when an indicator 40 was identified at the address associated with their account. For example, the processing unit 30 could issue a notification (e.g., an email or a text message) explaining that no waste container was found on their premises that day. In some embodiments, the notification may provide an option to the customer to arrange for supplemental pick-up service, which could be offered at an additional cost. The media could also be used to provide evidence in case a customer calls to dispute whether or not the waste container was properly accessible on the day of the waste removal route).
As per claim 20, Clifton and Mahan disclose the refuse vehicle of claim 16, wherein the instructions further cause the one or more processors to transmit a notification to a driver of the refuse vehicle (Mahan, ¶0056, The user interface generated by UI manager 422 may provide means for a user (e.g., an operator of refuse vehicle 10) to interact with refuse vehicle 10 and/or actuator assembly 436 for semi-autonomous or non-autonomous operations. For example, a user interface that indicates two or more refuse cans may provide means for the user to select a particular one of the refuse cans to act on (e.g., to move to and engage). The user interface may also provide other information regarding the operations of refuse vehicle 10, such as alarms, warnings, and or notifications. In some embodiments, the user interface generated by UI manager 422 may include a notification when a human being is detected within a danger zone. This may alert an operator to an unsafe condition and/or may indicate to the operator why automated refuse can collection cannot be implemented).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Primary Examiner, Art Unit 2668
Prosecution Insights