DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed 09/03/2025 has been entered. Claims 1, 3-11 and 13-20 are pending in the application.
Applicant’s amendment overcomes the 35 U.S.C. 112(b) rejections from the previously filed Office Action.
Applicant’s amendment overcomes the claim objections from the previously filed Office Action.
Response to Arguments
Applicant's arguments filed 09/03/2025 have been fully considered but they are not persuasive.
In regards to the independent claims, Applicant argues on pages 4-5 of the Remarks:
“In Nichols, the integrated radar sensor is the only element of the invention that may be used for analysis of one or more objects. The invention of Nichols is directed to a method of mapping the radar sensor data points to positions in image data recorded by an imaging device. The image data is essentially passive in the process.
In contrast, the Applicant’s invention actively uses data from the radar and the imaging device to identify moving targets and to determine a 3D location of the moving target. There is nothing in Nichols that teaches or suggests fusing data from the radar and the imaging device in this way. There is no suggestion that the radar only provides azimuth and range data while the imaging device only provides azimuth and elevation data, and that the data is fused into azimuth, range and elevation data based on matching azimuth data. The section referred to by the Examiner (Col. 11, lines 32-37) states: “According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.” This describes correlating the image data and the spatial model, it does not describe fusing radar data and image data to produce a 3D location of a moving target. Specifically, Nichols teaches using elevation and azimuth (which come from the radar data) with reference to a fixed data point (which is the location of the device). It does not teach using azimuth and elevation from an imaging device.”.
The Examiner respectfully disagrees. Based on the broadest reasonable interpretation of the claim language, the detection of the moving target using radar and matching that data with obtained image data indeed fulfills the limitation of “identifies moving radar targets and moving image targets having matching azimuth data as the moving target”. A “moving image target” is not a target that is actively moving in image data, rather just a “moving target” that is captured by image data. A target that is “moving” and captured by image data is “a moving image target”. The Applicant’s argument that the image data in Nichols is “passive” data is therefore found to be unpersuasive. In regards to the limitation, "fuses azimuth and range data from the Doppler radar with azimuth and elevation data from the imaging device and generates azimuth, range and elevation data of the moving target", the Examiner finds the Applicant’s arguments to be unpersuasive. Nichols discloses in col. 22, lines 28-37, “Generating the spatial model at block 215 may include correlating data points of the spatial model to the captured image data and assigning geospatial data to one or more portions of the image data. According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”. The spatial model data points are indeed the radar data points of azimuth, elevation and range. When correlating the data points of the spatial model with the captured image data the data points are indeed “fused” spatial data points based on the elevation, azimuth and reference to a fixed data point (i.e. “range data”). The Examiner believes that the Applicant is interpreting the claim language more narrowly than the broadest reasonable interpretation of the limitations.
Therefore, the Examiner maintains the rejection from the previously filed Office Action where Noon et al. (US 20130120182 A1) in view of Nichols et al. (US 10254395 B2) further in view of Avignon et al. (US 20140036085 A1) discloses the claimed invention of claim 1. Claim 10 is a broader version of claim 1 and therefore the same arguments presented for claim 1 above are applied to claim 10.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1,3-11 and 13-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Noon et al. (US 20130120182 A1), hereinafter Noon, in view of Nichols et al. (US 10254395 B2), hereinafter Nichols, further in view of Avignon et al. (US 20140036085 A1), hereinafter Avignon.
Regarding claim 1, Noon discloses [Note: what Noon fails to clearly disclose is strike-through]
A slope failure monitoring system (see system of Fig. 1, Work area monitor 10 which performs slope failure monitoring, further see paragraph 0047, “Referring to FIG. 1 there is shown a first embodiment of a Work Area Monitor 10. The Work Area Monitor 10 comprises a radar module 11 and (optionally) a camera 12 mounted on the tray of a utility motor vehicle 13 or otherwise associated with the radar module 11. A processor (not visible) is located in the cabin 14 together with the display screen 23.”, further see paragraph 0053, “The Work Area Monitor 10 is a comparatively short range device that provides movement monitoring of slopes at a range of about 30 metres to about 200 metres.”) comprising:
a two-dimensional (2D) (see Fig. 1, radar 11, further see paragraph 0053, “In operation the utility motor vehicle 13 is located so that the radar 11 and camera 12 point at a section of slope or wall to be monitored.”, Note: the radar 11 performs azimuth and range detection and therefore is a “2D radar”) that acquires azimuth and range data of moving radar targets in a scene (see paragraph 0051, “The synthetic image is able to display salient features in the field of view which a user is able to use for directing the Work Area Monitor. The synthetic image may be a digital terrain map, which is a 3D image comprising azimuth, elevation and range of each pixel measured by the radar.”);
a 2D (see Fig. 1, camera 12, further see paragraph 0053, “In operation the utility motor vehicle 13 is located so that the radar 11 and camera 12 point at a section of slope or wall to be monitored.”, where a camera operates on an optical frequency band) that acquires(see paragraph 0051, “An example of a digital terrain map is shown in the lower part of FIG. 16. The corresponding visual image formed from a composite of photographs is shown in the top part of FIG. 16.”, where the camera image acquires movement data through sequential scans, further see paragraph 0016, “The camera preferably takes a new image during every radar scan so any movement of the slope measured by the radar and the location of the movement is visually captured at the time of detection. The camera may have low light capability to work effectively at night.”), wherein the 2D (see Fig. 1, where radar 11 and camera 12 are co-located on the motor vehicle 13), having a common origin and a common line-of-sight (see paragraph 0050, “It is not always practical for the vehicle to be reversed into position so that the camera and radar point at the monitored wall over the back of the vehicle. Therefore, in one alternate embodiment, the camera is configured to be rotatable within a 180 degree field of view centred over the rear of the vehicle. The centre of the radar field of view is aligned with the centre of the camera field of view so that the image viewed on the screen 23 represents what will be monitored by the radar.”), such that moving radar targets and moving image targets having matching azimuth data are identifiable as a moving target (see paragraph 0050, “It is not always practical for the vehicle to be reversed into position so that the camera and radar point at the monitored wall over the back of the vehicle. Therefore, in one alternate embodiment, the camera is configured to be rotatable within a 180 degree field of view centred over the rear of the vehicle. The centre of the radar field of view is aligned with the centre of the camera field of view so that the image viewed on the screen 23 represents what will be monitored by the radar.”: NOTE: the BRI of “identifiable” simply indicates that the screen displays data which displays the “moving radar targets” and “moving image targets” as such are “identifiable” as a moving target);
a processing unit (see paragraph 0047, “A processor (not visible) is located in the cabin 14 together with the display screen 23.”) that processes azimuth and range data from the (see paragraph 0048, “A computer interface connects the processor to the radar module 11, camera 12 and display screen 23.”, further see paragraph 0053, “The Work Area Monitor 10 is a comparatively short range device that provides movement monitoring of slopes at a range of about 30 metres to about 200 metres. In operation the utility motor vehicle 13 is located so that the radar 11 and camera 12 point at a section of slope or wall to be monitored. An image of the slope as seen by the camera 12 is displayed on the screen 23. The radar dish 20 is aligned with the field of view of the camera 12 during manufacture so that it is known, to within an acceptable degree of accuracy, that the centre of the field of view of the radar corresponds with the centre of the field of view of the camera.”, where the camera and radar are aligned to the slope or wall to be monitored and therefore, the processing unit processes the radar data (i.e. azimuth and range data) and the camera data (i.e. azimuth and elevation data) to display the area as shown in Fig. 5, where the camera and radar data are overlayed) and:
determines a three-dimensional (3D) location of a moving target in the scene (paragraph 0051, “The synthetic image may be a digital terrain map, which is a 3D image comprising azimuth, elevation and range of each pixel measured by the radar. An example of a digital terrain map is shown in the lower part of FIG. 16. The corresponding visual image formed from a composite of photographs is shown in the top part of FIG. 16”, further see paragraph 0053, “The Work Area Monitor 10 is a comparatively short range device that provides movement monitoring of slopes at a range of about 30 metres to about 200 metres.”);
a display (see Fig. 2, display 23) that shows at least the scene and the location of the moving target in the scene (see paragraph 0075, “In the absence of any detected movement the screen 23 will display the work area, as shown in FIG. 9. In one embodiment, if the Work Area Monitor 10 detects movement within the selected work area the pixels that have movement are highlighted, for example by rendering the pixel boundaries in yellow as shown in FIG. 10.”); and
an alarm unit that generates an alarm when movement of the moving target is detected according to criteria (see paragraph 0077, “It will be noted that the alarm conditions can be reset or changed by selecting either option through clicking the appropriate button shown in FIG. 12. Selecting either option takes the user to an alert screen shown in FIG. 13. On the alert screen a user can adjust various alarm conditions such as the movement threshold (in millimetres), the number of moving cells required to trigger an alarm, the number of `looks` that need to show the movement to trigger the alarm and the direction of movement. The alert screen also may display an alert history in the current location to assist the user to determine if the alert is significant or possibly a false alarm. The alert level can also be set. For instance, FIG. 13 shows a configuration where a yellow alarm is notified to local crew but a red alarm is notified to a supervisor, as well as local crew.”).
Nichols discloses [Note: what Nichols fails to disclose is strike-through],
a 2D (see Col. 8, lines 16-21, “As shown in FIG. 1B, device 135 includes image module 140, and display 145. Imaging module 140 may include one or more imaging sensors for detecting image and/or video data. Image sensors of imaging module 140 may relate to one or more cameras or imaging devices configured to detect image or video data.”, where a camera operated on an optical frequency band) that acquires azimuth and elevation data of moving image targets in the scene (Col. 11, lines 32-37, “According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, where the correlation between the image data and the radar data based on elevation, azimuth and a reference to a fixed data point means that “azimuth and elevation data” is acquired for both the image data and radar data),
a processing unit that processes azimuth and range data from the Doppler radar and azimuth and elevation data from the imaging device (see Fig. 1A and Fig. 1B, where processors 130 and 105 are used to generate the spatial model based matching data points between the radar and image data, further see Col. 11, lines 32-37) and:
identifies moving radar targets and moving image targets having matching azimuth data as the moving target (Col. 11, lines 32-37, “According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, further see claim 9 for support, further see Col. 15, lines 14-25 where the matching of the data points is to track the object data on the display over-time and therefore to identify moving targets);
fuses azimuth and range data from the Doppler radar (see Col. 7, lines 9-12 which discloses that the radar generates doppler information and therefore is a “doppler radar”) with azimuth and elevation data from the imaging device and generates azimuth, range and elevation data of the moving target (the fusing of the radar and image data is disclosed in col. 22, lines 28-37, “Generating the spatial model at block 215 may include correlating data points of the spatial model to the captured image data and assigning geospatial data to one or more portions of the image data. According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, where a reference to a fixed data point in a spatial map is “range data”, further see claim 9 for support, further see for support Col. 13, lines 6-8, “Each grid spacing in FIG. 3B may be assigned a one or more positional values, such as distance, azimuth, and coordinates, for generating a spatial model.”, where the correlation is between the radar data detected by device 100 (see Fig. 1A) and the image data captured by device 135 (see Fig. 1B));
It would have been obvious to someone with ordinary skill in the art prior to the
effective filing date of the claimed invention to incorporate the features as disclosed by Nichols into the invention of Noon. Both Noon and Nichols are considered analogous arts to the claimed invention as they both disclose the fusion of radar and camera data for object detection. Noon discloses the use of a radar and a camera as shown in the citations above; however, fails to clearly disclose the feature of the radar to be a doppler radar and the camera to be a high-definition camera. Nichols discloses the feature of using a doppler-measuring radar (i.e. doppler radar) for object detection and further discloses the features of matching the radar and image data points in azimuth, elevation and range. The combination of Noon and Nichols would be obvious with a reasonable expectation of success in order to fuse radar and image data to increase resolution of the data while meeting requirements for a low cost device (see paragraph 0031 of Nichols).
Avignon discloses,
the imaging device is a high definition imaging device (see paragraph 0026, “The head 12 incorporates the detectors required for presence detection in the area to be monitored, which can take the form of one or several cameras comprising high definition identification equipment such as a high resolution camera.”).
It would have been obvious to someone with ordinary skill in the art prior to the
effective filing date of the claimed invention to incorporate the features as disclosed by Avignon into the invention of Noon in view of Nichols. Noon and Avignon are considered analogous arts as they both disclose the use of radar and camera devices on a vehicle for slope monitoring of a terrain. All three references are considered analogous arts to the claimed invention as they all disclose the fusion of radar and camera data for object detection. Noon and Nichols fail to clearly disclose that the imaging camera is a high-resolution imaging camera. Avignon discloses the use of radar and camera sensors on a vehicle for slope monitoring of a terrain and further discloses that the camera can be a high-definition camera. The combination of Noon and Nichols and Avignon would be obvious with a reasonable expectation of success in order to provide high resolution imaging data for the system (see paragraph 0026 of Avignon).
Regarding claim 3, the combination of Noon, Nichols and Avignon discloses,
The slope failure monitoring system of claim 1. Noon further discloses [Note: what Noon fails to disclose is strike-through], wherein the 2D (see paragraph 0048, “The radar module 11 is a scanning dish device that consists of a dish 20 mounted on a scanning gimbal 21 that has a vertical scan of -10 to +40 degrees and a horizontal scan of -55 to +55 degrees. The gimbal houses a 600 mm parabolic dish with an offset feed. The antenna transmits and receives radio frequency signals in X-band with T/R gating to separate the direct path from the wall reflection and sufficient range resolution to separate foreground anomalies (such as mining vehicles) from the wall reflections.”).
See cited section and motivation of claim 1 above for combining Noon and Nichols and Avignon to utilize a doppler radar in the invention of Noon.
Regarding claim 4, the combination of Noon, Nichols and Avignon discloses,
The slope failure monitoring system of claim 1. Noon further discloses [Note: what Noon fails to disclose is strike-through], wherein the 2D (see paragraph 0016, “The camera is suitably a digital camera capable of recording sequential still images or video images. The camera is preferably mounted on the vehicle separate from the radar module. Alignment between the radar field of view and the camera field of view is preferably set up during manufacture. The camera preferably takes a new image during every radar scan so any movement of the slope measured by the radar and the location of the movement is visually captured at the time of detection.”).
See cited section and motivation of claim 1 above for combining Noon and Nichols and Avignon to utilize a high-definition imaging device in the invention of Noon.
Regarding claim 5, the combination of Noon, Nichols and Avignon discloses,
The slope failure monitoring system of claim 1. Noon further discloses [Note: what Noon fails to disclose is strike-through], wherein the processing unit is a single device that performs all required processing of data obtained from the (see paragraph 0047, “A processor (not visible) is located in the cabin 14 together with the display screen 23.”, further see paragraph 0048, “The control for scanning the gimbal is contained in the cabin 14. A computer interface connects the processor to the radar module 11, camera 12 and display screen 23”).
See cited section and motivation of claim 1 above for combining Noon and Nichols and Avignon to utilize a doppler radar and high-definition camera in the invention of Noon.
Regarding claim 6, the combination of Noon, Nichols and Avignon discloses,
The slope failure monitoring system of claim 1.
Noon further discloses [Note: what Noon fails to disclose is strike-through],
wherein the processing unit comprises multiple devices (see paragraph 0019, “The synthetic image may be generated by the processor, or another processor specific to the application.”, where the use of multiple processors is disclosed (i.e. “the processor” and “another processor specific to the application”) that (paragraph 0051, “The synthetic image may be a digital terrain map, which is a 3D image comprising azimuth, elevation and range of each pixel measured by the radar. An example of a digital terrain map is shown in the lower part of FIG. 16. The corresponding visual image formed from a composite of photographs is shown in the top part of FIG. 16.”, further see paragraph 0053, “The Work Area Monitor 10 is a comparatively short range device that provides movement monitoring of slopes at a range of about 30 metres to about 200 metres.”), fuses data to determine the 3D location of the moving target (paragraph 0051, “The synthetic image may be a digital terrain map, which is a 3D image comprising azimuth, elevation and range of each pixel measured by the radar. An example of a digital terrain map is shown in the lower part of FIG. 16. The corresponding visual image formed from a composite of photographs is shown in the top part of FIG. 16.”, further see paragraph 0053, “The Work Area Monitor 10 is a comparatively short range device that provides movement monitoring of slopes at a range of about 30 metres to about 200 metres.”, where “fuses data” implies any data fused together to determine the 3D location of the slope (i.e. the slope is the moving target being detected to determine whether the movement above a threshold has occurred)), and applies threshold criteria to generate the alarm (see paragraph 0077, “It will be noted that the alarm conditions can be reset or changed by selecting either option through clicking the appropriate button shown in FIG. 12. Selecting either option takes the user to an alert screen shown in FIG. 13. On the alert screen a user can adjust various alarm conditions such as the movement threshold (in millimetres), the number of moving cells required to trigger an alarm, the number of `looks` that need to show the movement to trigger the alarm and the direction of movement. The alert screen also may display an alert history in the current location to assist the user to determine if the alert is significant or possibly a false alarm. The alert level can also be set. For instance, FIG. 13 shows a configuration where a yellow alarm is notified to local crew but a red alarm is notified to a supervisor, as well as local crew.”).
Nichols discloses [Note: what Nichols fails to disclose is strike-through],
wherein the processing unit (see Figs. 1A and 1B where the processing unit (i.e. 105)) process azimuth and range data from the 2D Doppler radar, azimuth and elevation data from the 2D (Col. 11, lines 32-37, “According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, where the correlation between the image data and the radar data based on elevation, azimuth and a reference to a fixed data point means that “azimuth and elevation data” is acquired for both the image data and radar data, further see Col. 7, lines 9-12 which discloses that the radar generates doppler information and therefore is a “doppler radar).
Avignon discloses,
the imaging device is a high definition imaging device (see paragraph 0026, “The head 12 incorporates the detectors required for presence detection in the area to be monitored, which can take the form of one or several cameras comprising high definition identification equipment such as a high resolution camera.”).
See cited section and motivation of claim 1 above for combining Noon and Nichols and Avignon to disclose the fusing of range, azimuth and elevation data from a camera and a radar. Further see the cited section and motivation of claim 1 to utilize a doppler radar and high-definition camera in the invention of Noon.
Regarding claim 7, Noon further discloses
The slope failure monitoring system of claim 1 wherein the criteria are various threshold requirements selected from at least one of: movement within a defined area (paragraphs 0060-0062, “An alarm is generated by setting a threshold for detected movement. If the radar processing results in a detected movement of greater than the set threshold the Work Area Monitor generates an audible and visible alarm that warns workers that they should vacate the area. In addition, the moving area may be highlighted on the screen 23 using a colour code. FIG. 7 shows the output of the screen 23 where a wall region with red overlay has moved. Another preferred option is to flash the pixels that show movement above the threshold…The Work Area Monitor triggers an alarm based on the following parameters: Threshold: The movement in `mm` of the work area to be considered as dangerous;”, further see paragraphs 0075-0076, “FIG. 11 shows a central few pixels that have larger or prolonged movement and are therefore highlighted in red. Adjacent pixels with large movement that is above the threshold may be shown in orange or red with small movement shown in yellow. Other schemes for displaying movement would also be suitable…If the movement exceeds a threshold an alarm is triggered, such as displayed in FIG. 12.”); movement occurring above a set velocity; and moving targets above a set size.
Regarding claim 8, Noon further discloses
The slope failure monitoring system of claim 1 further comprising an Input Device for a User to input filters selected from at least one of: radar data mask (paragraph 0078, “The user can also elect to add a mask from the alert screen. This may be required, for instance, if movement is apparently due to a non-slope artefact such as the drill mast referred to earlier. Clicking the "edit mask" button takes the user to the "edit alarm mask" screen shown in FIG. 14. From this screen the user can mask out certain areas using common tools as shown across the top of the screen shot. Once selected the screen is exited by selecting "ok".”); radar spatial alarm zone (paragraph 0078, “The user can also elect to add a mask from the alert screen. This may be required, for instance, if movement is apparently due to a non-slope artefact such as the drill mast referred to earlier. Clicking the "edit mask" button takes the user to the "edit alarm mask" screen shown in FIG. 14. From this screen the user can mask out certain areas using common tools as shown across the top of the screen shot. Once selected the screen is exited by selecting "ok".”); image data mask (see paragraph 0066, “If the work area contains vegetation or machinery a false alarm may be generated. Areas that may generate a false alarm can be masked. An `Edit Mask` mode is selected and a green mask is drawn over the problem areas as described in more detail below.”); and image data spatial alarm zone (paragraph 0078, “The user can also elect to add a mask from the alert screen. This may be required, for instance, if movement is apparently due to a non-slope artefact such as the drill mast referred to earlier. Clicking the "edit mask" button takes the user to the "edit alarm mask" screen shown in FIG. 14. From this screen the user can mask out certain areas using common tools as shown across the top of the screen shot. Once selected the screen is exited by selecting "ok".”).
Regarding claim 9, Noon further discloses
The slope failure monitoring system of claim 1 further comprising an Input Device for a User to input threshold criteria selected from at least one: Radar target speed; Radar target bearing; Radar Cross Section; Radar target Azimuth and/or Range filter (see paragraphs 0060-0062, “An alarm is generated by setting a threshold for detected movement. If the radar processing results in a detected movement of greater than the set threshold the Work Area Monitor generates an audible and visible alarm that warns workers that they should vacate the area. In addition, the moving area may be highlighted on the screen 23 using a colour code. FIG. 7 shows the output of the screen 23 where a wall region with red overlay has moved. Another preferred option is to flash the pixels that show movement above the threshold…The Work Area Monitor triggers an alarm based on the following parameters: Threshold: The movement in `mm` of the work area to be considered as dangerous;”, where movement within a millimeter threshold is a range filter); Multiple radar target; Radar temporal hysteresis; Image data angular speed; Image data target elevation and/or azimuth size; and Image target classification.
Regarding claim 10, Noon discloses [Note: what Noon fails to clearly disclose is strike-through]
A method of monitoring a slope for failure (see system of Fig. 1, Work area monitor 10 which performs slope failure monitoring, further see paragraph 0047, “Referring to FIG. 1 there is shown a first embodiment of a Work Area Monitor 10. The Work Area Monitor 10 comprises a radar module 11 and (optionally) a camera 12 mounted on the tray of a utility motor vehicle 13 or otherwise associated with the radar module 11. A processor (not visible) is located in the cabin 14 together with the display screen 23.”, further see paragraph 0053, “The Work Area Monitor 10 is a comparatively short range device that provides movement monitoring of slopes at a range of about 30 metres to about 200 metres.”), including the steps of:
co-locating a two-dimensional (2D) (see Fig. 1, where radar 11 and camera 12 are co-located on the motor vehicle 13, where both the radar 11 and camera 12 determine two-dimensional data) with a shared or overlapping field of view of a scene (see paragraph 0050, “It is not always practical for the vehicle to be reversed into position so that the camera and radar point at the monitored wall over the back of the vehicle. Therefore, in one alternate embodiment, the camera is configured to be rotatable within a 180 degree field of view centred over the rear of the vehicle. The centre of the radar field of view is aligned with the centre of the camera field of view so that the image viewed on the screen 23 represents what will be monitored by the radar.”); calibrating the (see paragraph 0050, “It is not always practical for the vehicle to be reversed into position so that the camera and radar point at the monitored wall over the back of the vehicle. Therefore, in one alternate embodiment, the camera is configured to be rotatable within a 180 degree field of view centred over the rear of the vehicle. The centre of the radar field of view is aligned with the centre of the camera field of view so that the image viewed on the screen 23 represents what will be monitored by the radar.”);
synchronising timing of data collection and processing of data collected from the (see paragraph 0016, “The Work Area Monitor may also include a camera. The camera is suitably a digital camera capable of recording sequential still images or video images. The camera is preferably mounted on the vehicle separate from the radar module. Alignment between the radar field of view and the camera field of view is preferably set up during manufacture. The camera preferably takes a new image during every radar scan so any movement of the slope measured by the radar and the location of the movement is visually captured at the time of detection. The camera may have low light capability to work effectively at night.”) on one or more processing units using detection and tracking algorithms to detect common moving targets identified by the (see displays of Fig 5, which depicts the processed radar and image data, further see paragraph 0032, “FIG. 5 indicates how the selected radar scan and pixels are overlayed on the camera's field of view of a slope;”; Note: the purpose of the overlay is to detect movement data of the slope),
raising an alarm if a common moving target satisfies one or more criteria (see paragraph 0077, “It will be noted that the alarm conditions can be reset or changed by selecting either option through clicking the appropriate button shown in FIG. 12. Selecting either option takes the user to an alert screen shown in FIG. 13. On the alert screen a user can adjust various alarm conditions such as the movement threshold (in millimetres), the number of moving cells required to trigger an alarm, the number of `looks` that need to show the movement to trigger the alarm and the direction of movement. The alert screen also may display an alert history in the current location to assist the user to determine if the alert is significant or possibly a false alarm. The alert level can also be set. For instance, FIG. 13 shows a configuration where a yellow alarm is notified to local crew but a red alarm is notified to a supervisor, as well as local crew.”).
Nichols discloses [Note: what Nichols fails to disclose is strike-through],
co-locating a 2D Doppler radar (see Fig. 1A, radar sensor 115 is “a 2D doppler radar”,further see Col. 7, lines 9-12, “According to one or more embodiments, radar sensor 115 may produce bearing, range and doppler data for objects in the field of view of the sensor, without reliance on ambient light, or susceptibility to difficult lighting conditions.”) and a 2D (see Fig. 1B, device 135 is a imaging device) at a common origin with a shared or overlapping field of view of a scene (see Col. 16, lines 25-36, “. Based on the data points determined at block 810, the data points may be matched to one or more portions of image data detected by the device. Matching data points may be based on calibration of the radar sensor and the image sensor. According to another embodiment, matching data points at block 815 may be based on overlaying image data and spatial modeling data. Portions of the image data may be matched to portions of the spatial model. At block 820, data points may be assigned to the image data. For example, for each data point matched to image data, geospatial data determined for the data point may be assigned to a particular region, or the position of the data point in the image data.”),
wherein the step of detecting common moving targets includes identifying moving radar targets and moving image targets having matching azimuth data as a common moving target (Col. 11, lines 32-37, “According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, further see claim 9 for support, further see Col. 15, lines 14-25 where the matching of the data points is to track the object data on the display over-time and therefore to identify moving targets, further see where the fusing of the radar and image data is disclosed in col. 22, lines 28-37, “Generating the spatial model at block 215 may include correlating data points of the spatial model to the captured image data and assigning geospatial data to one or more portions of the image data. According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, further see for support Col. 13, lines 6-8, “Each grid spacing in FIG. 3B may be assigned a one or more positional values, such as distance, azimuth, and coordinates, for generating a spatial model.”, where the correlation is between the radar data detected by device 100 (see Fig. 1A) and the image data captured by device 135 (see Fig. 1B));
It would have been obvious to someone with ordinary skill in the art prior to the
effective filing date of the claimed invention to incorporate the features as disclosed by Nichols into the invention of Noon. Both Noon and Nichols are considered analogous arts to the claimed invention as they both disclose the fusion of radar and camera data for object detection. Noon discloses the use of a radar and a camera as shown in the citations above; however, fails to clearly disclose the feature of the radar to be a doppler radar and the camera to be a high-definition camera. Nichols discloses the feature of using a doppler-measuring radar (i.e. doppler radar) for object detection and further disclose the features of matching the radar and image data points in azimuth, elevation and range. The combination of Noon and Nichols would be obvious with a reasonable expectation of success in order to fuse radar and image data to increase resolution of the data while meeting requirements for a low cost device (see paragraph 0031 of Nichols).
Avignon discloses,
the imaging device is a high definition imaging device (see paragraph 0026, “The head 12 incorporates the detectors required for presence detection in the area to be monitored, which can take the form of one or several cameras comprising high definition identification equipment such as a high resolution camera.”).
It would have been obvious to someone with ordinary skill in the art prior to the
effective filing date of the claimed invention to incorporate the features as disclosed by Avignon into the invention of Noon in view of Nichols. Noon and Avignon are considered analogous arts as they both disclose the use of radar and camera devices on a vehicle for slope monitoring of a terrain. All three references are considered analogous arts to the claimed invention as they all disclose the fusion of radar and camera data for object detection. Noon and Nichols fail to clearly disclose that the imaging camera is a high-resolution imaging camera. Avignon disclose the use of radar and camera sensors on a vehicle for slope monitoring of a terrain and further discloses that the camera can be a high-definition camera. The combination of Noon and Nichols and Avignon would be obvious with a reasonable expectation of success in order to provide high resolution imaging data for the system (see paragraph 0026 of Avignon).
Regarding claim 11, Noon further discloses
The method of claim 10 further including the step of applying one or more filters to only raise an alarm that pass the filters (paragraph 0077, “It will be noted that the alarm conditions can be reset or changed by selecting either option through clicking the appropriate button shown in FIG. 12. Selecting either option takes the user to an alert screen shown in FIG. 13. On the alert screen a user can adjust various alarm conditions such as the movement threshold (in millimetres), the number of moving cells required to trigger an alarm, the number of `looks` that need to show the movement to trigger the alarm and the direction of movement.”).
Regarding claim 13, the combination of Noon and Nichols and Avignon discloses
The method of claim 10.
The combination of Noon and Avignon fails to disclose:
wherein matching azimuth data includes the steps of:
calculating a centroid of each tracked target in azimuth and range for the radar data;
calculating centroid of each tracked target in azimuth and elevation for the imaging device data; and
identifying tracked targets with shared or overlapping azimuth locations as targets with matching azimuth data.
Nichols discloses,
wherein matching azimuth data (see Col. 4, lines 26-32, “For example, data generated by a radar sensor and one or more imaging sensors may be used to generate spatial models, and tracking one or more objects. According to another embodiment, captured image data may be matched to data points detected by the radar sensor. Geospatial data may then be matched with image data according to one or more embodiments.”) includes the steps of:
calculating a centroid of each tracked target in azimuth and range for the radar data (see Fig. 5A, where a centroid of tracked target 526 is determined: Note: the spatial model is correlated with both the radar and image data and therefore the centroid is calculated for each target and therefore calculated for both the radar detected target and image detected target);
calculating centroid of each tracked target in azimuth and elevation for the imaging device data (see Fig. 5A, where a centroid of tracked target 526 is determined: Note: the spatial model is correlated with both the radar and image data and therefore the centroid is calculated for each target and therefore calculated for both the radar detected target and image detected target); and
identifying tracked targets with shared or overlapping azimuth locations as targets with matching azimuth data (Col. 11, lines 32-37, “According to one embodiment, image data and the spatial model are correlated based on one or more of elevation, azimuth, and reference to a fixed data point, such as a position of the device. Based on the spatial model generated at block 215, captured image data may be displayed to include data points of the spatial model.”, further see Col. 16, 37-43, “In certain embodiments, matching/correlating data points to image data includes assigning spatial model data to the image data. In another embodiment, matching/correlating data points to image data may include assigning only a portion of detected data points to the image data. At block 825, the device may store the image data including data points assigned to the image.”, further see claim 9 for support).
It would have been obvious to someone with ordinary skill in the art prior to the
effective filing date of the claimed invention to incorporate the features as disclosed by Nichols into the invention of Noon in view of Avignon. All three references are considered analogous arts to the claimed invention as they all disclose the fusion of radar and camera data for object detection. The combination would be obvious with a reasonable expectation of success in order to fuse radar and image data while meeting requirements for a low cost device (see paragraph 0031 of Nichols).
Regarding claim 14, Noon further discloses
The method of claim 13 further including defining a buffer zone to the tracked data for the radar target and defining a buffer zone to the tracked data for the imaging device target (see paragraph 0055, “An operator selects a region of wall to monitor. In the preferred embodiment the screen 23 is a touch screen and the operator selects the region by tracing the region on the touch screen. A grid of pixels is overlayed on the scene to display the area that will be monitored. The area can be adjusted by moving the corners of the area, as shown in FIG. 4. The user clicks `OK` on the work area selection screen to commence monitoring.”, where the area to be monitored is selected (i.e. defining a buffer zone) and this area is used for both aligning the radar and camera data) and identifying tracked targets with shared or overlapping azimuth locations anywhere within the buffer zone of both the tracked radar target and the tracked imaging device target (see Fig. 5 which depicts the overlaying of the radar and camera data points on the display of the selected area, further see paragraph 0032, “FIG. 5 indicates how the selected radar scan and pixels are overlayed on the camera's field of view of a slope”).
Regarding claim 15, Noon further discloses
The method of claim 14 wherein the buffer zone to the tracked data for the radar target is an angular degree (see paragraph 0055, “An operator selects a region of wall to monitor. In the preferred embodiment the screen 23 is a touch screen and the operator selects the region by tracing the region on the touch screen. A grid of pixels is overlayed on the scene to display the area that will be monitored. The area can be adjusted by moving the corners of the area, as shown in FIG. 4. The user clicks `OK` on the work area selection screen to commence monitoring.”, where selecting and moving the corners of the area to adjust and define the “buffer zone” is “an angular degree”).
Regarding claim 16, Noon further discloses
The method of claim 14 wherein the buffer zone to the tracked data for the imaging device target is a percentage of the size of the imaging device target (see paragraph 0055, “An operator selects a region of wall to monitor. In the preferred embodiment the screen 23 is a touch screen and the operator selects the region by tracing the region on the touch screen. A grid of pixels is overlayed on the scene to display the area that will be monitored. The area can be adjusted by moving the corners of the area, as shown in FIG. 4. The user clicks `OK` on the work area selection screen to commence monitoring.”, where selecting and moving the corners of the area to adjust and define the “buffer zone” to monitor the region of interest is “the buffer zone to the tracked data for the imaging device target is a percentage of the size of the target”, as the target (i.e. slope) is a percentage of the entire selected detection area).
Regarding claim 17, the cited section and motivation in claim 1 is applied.
Regarding claim 18, Noon further discloses
The method of claim 10 further including the step of displaying on a display device (see Fig. 2, display 23) at least the scene and the location of the common moving target in the scene (see paragraph 0075, “In the absence of any detected movement the screen 23 will display the work area, as shown in FIG. 9. In one embodiment, if the Work Area Monitor 10 detects movement within the selected work area the pixels that have movement are highlighted, for example by rendering the pixel boundaries in yellow as shown in FIG. 10.”).
Regarding claim 19, Noon further discloses
The method of claim 10 further including the step of displaying range indicators on a display device (see Fig. 10 which displays the movement in the scene and displays “range indicators” as noted by the pixel boundaries).
Regarding claim 20, Noon further discloses
The method of claim 10 wherein the imaging device is a video camera that records a sequence of optical images of a scene (see paragraph 0016, “The camera is suitably a digital camera capable of recording sequential still images or video images. The camera is preferably mounted on the vehicle separate from the radar module. Alignment between the radar field of view and the camera field of view is preferably set up during manufacture. The camera preferably takes a new image during every radar scan so any movement of the slope measured by the radar and the location of the movement is visually captured at the time of detection.”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the da