DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-10, as amended, are currently pending and have been considered below.
Claim Objections
Claim 1 is objected to because of the following informalities: Claim 1 recites the limitation, “whether the secondstacked frame” in line 25 of the claim. This limitation appears to contain a typographical error and should recite, i.e., “whether the second stacked frame.” Appropriate correction is required.
Claim 4 is objected to because of the following informalities: Claim 4 recites the limitation, “obtain a corresponding first set of shifted frame (Fsh)” in line 13 of the claim. This limitation appears to contain a typographical error and should recite, i.e., “obtain a corresponding first set of shifted frames (Fsh).” Appropriate correction is required.
Claim 4 is objected to because of the following informalities: Claim 4 recites the limitation, “stacking (30) said corresponding first set of shifted frame (Fsh) with the first frame (F0) of the subset of frames” in lines 14-15 of the claim. This limitation appears to contain a typographical error and should recite, i.e., “stacking (30) said corresponding first set of shifted frames (Fsh) with the first frame (F0) of the subset of frames.” Appropriate correction is required.
Claim 5 is objected to because of the following informalities: Claim 5 recites the limitation, “obtain a corresponding second set of shifted frame (Fsh)” in line 13 of the claim. This limitation appears to contain a typographical error and should recite, i.e., “obtain a corresponding second set of shifted frames (Fsh).” Appropriate correction is required.
Claim 5 is objected to because of the following informalities: Claim 5 recites the limitation, “stacking (70) said corresponding second set of shifted frame (Fsh) with the first frame (F0) of the set of frames” in lines 14-15 of the claim. This limitation appears to contain a typographical error and should recite, i.e., “stacking (70) said corresponding second set of shifted frame (Fsh) with the first frame (F0) of the set of frames.” Appropriate correction is required.
Claim 8 is objected to because of the following informalities: Claim 8 recites the limitation, “System … configured to carry out the method according to claim 7” in lines 1-3 of the claim. The “system” of claim 8 does not comprise any components, i.e., memory, processor, storage, to carry out the method of claim 7. Appropriate correction is required.
Claims 9 and 10 are objected to because of the following informalities: Claim 9 recites the limitation, “Software loadable in a system … and configured to allow, when run, the system to implement the method according claim 1” in lines 1-3 of the claim. The “system” of claim 9 does not comprise any components, i.e., memory, processor, storage, to carry out the method of claim 1. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shao et al., U.S. Publication No. 2017/0358103, hereinafter, “Shao”, and further in view of Tchilian, U.S. Publication No. 2020/0174094, hereinafter, “Tchilian”.
As per claim 1, Shao discloses a computer-implemented method for detecting objects in the field of view of an optical detection device (2) (Shao, ¶0007, Execution of the object tracking application directs the processor to receive a sequence of images, wherein at least one moving object is visible in relation to a background in the sequence of images; Shao, ¶0008, the object tracking system further comprises a camera in communication with the processor) comprising:
acquiring (10), through the optical detection device (2), a plurality of subsequent frames (F) of a portion of an observed space in a time interval (T), wherein each frame (F) is acquired in a corresponding time instant (Tk) of the time interval (T) (Shao, ¶0008, the object tracking system further comprises a camera in communication with the processor; Shao, ¶0070, According to several embodiments of the invention, an object tracking system applies a shift-and-add process to a sequence of short high-frame rate exposures of a scene, to synthetically render a long-exposure image of a moving object within the scene; Shao, ¶0074, an object tracking system 200 such as that illustrated in FIG. 2 receives a sequence of images at processing system 210. The images may include consecutive exposures of a scene 100);
for each set of values of shifting parameters (|(νM)|, θN) of a plurality of sets of values of shifting parameters (|(νM)|, θN) that are indicative corresponding predetermined motions of an object with respect to the optical detection device (2), determining (30) a corresponding first stacked frame (Fs, Fs,g, Fs,b) on the basis of the subset of frames (Fsel) and as a function of the set of values of the shifting parameters (|(νM)|, θN) (Shao, ¶0076, Although the moving object 110 may appear faint or undetectable within a single frame, according to a number of embodiments of the invention, processing system 210 performs an object tracking process that can include a series of shift-and-add computations. These shift-and-add computations can be conceptually viewed as a processing in which the images within the sequence are shifted so that the moving object 110 within one image is matched up with the position of the moving object 110 within other images. Upon summing the pixels of the shifted images, the moving object 110 may appear more prominently in this synthetically rendered long-exposure image. A conceptual illustration of such shifted and added frames 500, with the object lined up at target track 510, is shown in FIG. 5; Shao, ¶0077, As can readily be appreciated, the location of moving objects in the images is not known a priori. Accordingly, the shift-and-add computations involve adopting a hypothesis that a given pixel within a frame images an object that is moving, and summing pixels across a sequence of preceding images corresponding to locations that would be occupied assuming that the object is moving at a given velocity. Each possible velocity (i.e. speed and direction) involves the generation of a different sum of pixel locations across the sequence of images. The specific pixels that are summed can be determined based upon shifts in pixel locations between successive frames in the sequence corresponding to a specific pixel per frame interval velocity);
for each first stacked frame (Fs, Fs,g, Fs,b), determining (40, 50, 60) whether the first stacked frame (Fs, Fs,g, Fs,b) belongs to a first or a second class (C1, C2), wherein the first and the second class are respectively indicative of the presence or absence of an image of an object in the first stacked frame (F.subs, Fs,g, Fs,b) (Shao, ¶0077, moving objects and their velocities are detected using sums of corresponding pixel locations in the sequence of images at specific velocities that result in values that exceed a predetermined threshold. When background subtraction is applied during the summing process, the correct velocity will yield a sum in which a high contrast pixel value is reinforced. The sums of incorrect velocity hypotheses will include many pixels having a background subtracted intensity (i.e. pixels that do not image the moving object) and so will result in a sum that is lower and less likely to exceed the threshold. When the sums generated for a given pixel location do not exceed the threshold at any hypothetical velocity, the processing system 210 can determine that the pixel location does not image a moving object); and
selecting, among said sets of values of shifting parameters (|(νg)|, θg), each set of values of shifting parameters that is associated to corresponding first stacked frame (Fs, Fs,g) classified as belonging to the first class (C1) (Shao, ¶0077, As can readily be appreciated, the location of moving objects in the images is not known a priori. Accordingly, the shift-and-add computations involve adopting a hypothesis that a given pixel within a frame images an object that is moving, and summing pixels across a sequence of preceding images corresponding to locations that would be occupied assuming that the object is moving at a given velocity. Each possible velocity (i.e. speed and direction) involves the generation of a different sum of pixel locations across the sequence of images. The specific pixels that are summed can be determined based upon shifts in pixel locations between successive frames in the sequence corresponding to a specific pixel per frame interval velocity. In practice, the shifts and sum process can result in a single object causing a number of pixel sums corresponding to different velocities. The pixel sum with the highest intensity can provide the best estimate of the true location and velocity of an object),
the method further comprising, for each selected set of values of shifting parameters (|(νg)|, θg): determining (70) a corresponding second stacked frame (Fs,n, Fs,n,g, Fs,n,b) on the basis of the set of frames (F) and as a function of the selected set of values of shifting parameters (|(νg)|, θg) (Shao, ¶0098, The result of performing shift-and-add calculations at each pixel location within a region of an image with respect to each of a number of different velocities is conceptually illustrated in FIG. 11. Each image in FIG. 11 corresponds to the summed intensities at a given pixel location, generated using a specific per frame shift relative to the pixel location to select pixels from a sequence of images, and then summing the selected pixels. The images correspond to different per frame integer pixel shifts in the x and/or y direction … the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined);
for each second stacked frame (Fs,n, Fs,n,g, Fs,n,b), determining (80, 90, 100) whether the second stacked frame (Fs,n, Fs,n,g, Fs,n,b) belongs to a third or a fourth class (C1, C2) wherein the third and the fourth class are respectively indicative of the presence or absence of an image of an object in the second stacked frame (Fs,n, Fs,n,g, Fs,n,b) (Shao, ¶0099, When pixel values are summed, those pixels imaging a moving object will appear as high-contrast pixels in relation to the other background-subtracted pixels in the image. According to several embodiments of the invention, summed intensity values from the sets of summed intensity values exceeding a threshold may be identified (410). A location of at least one moving object in the reference image may also be identified based on a summed intensity value from a set of summed intensity values exceeding a threshold; Shao, ¶0100, The high summed intensity values can thus include more than one peak, from which the moving object's position and velocity may be estimated, in accordance with some embodiments of the invention; Shao, ¶0106, In an object tracking process for space debris, as an example, numerous velocities may be examined. Given a multiple-image data set, applying an object tracking application according to an embodiment of the invention to each of the velocities results in a number of synthetic images. As a single bright moving object may result in many above-threshold pixels, the output of the shift-and-add process may produce numerous pixels that are above threshold. At this point, it may be unclear as to whether there exist a significant number of moving objects, a single moving object, or a number in between; Shao, ¶0107, A clustering method in accordance with many embodiments of the invention can sort through the clutter and arrive at the correct number of moving objects in the scene; Shao, ¶0110, a fractional pixel interpolation operation may be used find the final best fit position and velocity after the clustering operation. The output of the clustering method can be sent to a non-linear least squares fitting routine that performs fractional pixel shift-and-add to find the position and velocity of the object);
determining (110) if the first stacked frame (Fs, Fs,g) classified as belonging to the first class (C1) and in the second stacked frame (Fs,n, Fs,n,g, Fs,n,b) classified as belonging to the third class (C1) meet a requirement (Shao, ¶0100, The high summed intensity values can thus include more than one peak, from which the moving object's position and velocity may be estimated … the position and/or velocity can be refined at a sub-pixel level by determining a sub-pixel shift based on the pixel offsets associated with the plurality of peaks. The sub-pixel shift may be determined by weighting the peak pixel offsets, or by assuming per frame shifts with sub-pixel components. Interpolation can then be used to generate intensity values at the sub-pixel locations. These values can then be summed to generate a summed intensity value for the sub-pixel shift. The result should yield the highest intensity peak, with its value estimating a refined velocity of the moving object); and
if said requirement is met, indicate (120) the presence of an object, wherein each of said predetermined motions has a corresponding trajectory and a corresponding speed, and wherein each of said set of values of the shifting parameters (|(νM)|, θN) comprise values indicating the trajectory and the speed of the corresponding motion (Shao, a thumbnail of the moving object may be generated and displayed or reported to a display device in real time. Some embodiments of the invention may also compute and report the object's size, position and/or velocity ... The thumbnail, in combination with other information such as velocity and trajectory, can also be utilized to classify the object as a certain type of object, or into a certain category. Certain embodiments of the invention generate an alert based on the classification of the moving object ... Examples of classifications include, but are not limited to, a UAV, asteroid, missile).
Shao does not explicitly disclose the following limitations as further recited however Tchilian discloses
selecting (20) a subset of frames (Fsel) from the plurality of frames (F) (Tchilian, ¶0008, Multiple mode star tracker devices and methods in accordance with embodiments of the present disclosure provide for … the detection of dim objects within an image area. The image sensor of the multiple mode star tracker features a global shutter, ensuring that each pixel of the sensor integrates signal for the same absolute time period, allowing for the precise combining or stacking of multiple image frames obtained by the image sensor ... Postprocessing of multiple video frames, where each pixel is registered to an IRF, further allows stacking of these frames in order to significantly boost signal-to-noise ratio (SNR). Through this process, multiple frames can be stacked, enabling the detection of very dim objects; Tchilian, ¶0010, the image data from many individual image frames can be accurately combined or stacked, enabling dim objects within the field of view of the multiple image frames to become visible; Tchilian, ¶0017, the multiple mode star tracker 108 enables stacking of multiple image frames in order to significantly boost the signal-to-noise ratio (SNR) of the device, allowing the detection of dim objects 120, such as a distant star, or some other object, such as a space craft, space junk, a meteoroid, or any other object within a field of view 116 that is not visible or that is not distinct within a single frame of image data .. star tracker 108 as described herein can provide position information at the same time that it collects image information; Tchilian, ¶0021, in connection with operation of the multiple mode star tracker 108 to obtain image information, a sequence of images can be obtained at some minimum frame rate. As an example, but without limitation, the minimum imaging frame rate may be 10 Hz or greater. Moreover, in order to detect very dim objects, some minimum number of frames can be collected. As an example, but without limitation, from 20 to 2000 frames of image data can be collected. Accordingly, at step 324, a determination can be made as to whether a minimum number of image frames have been collected. The minimum number of image frames can be a fixed value, or can be variable, for example dependent upon a desired sensitivity level or sets of sensitivity levels. As still another example, the minimum number of frames can be determined dynamically. For instance, a neural network, human observer, threshold detector, or other control or process can determine the minimum number of frames based on whether a dim object 120 becomes visible; Tchilian, ¶0024, the analysis process can operate the multiple mode star tracker 108 to aggregate additional frames 504 of image data to create one or more additional composite images 508. Such additional composite images 508 can include the original composite image 508, or can be comprised of data from the image frames 504 collected subsequent to the image frames 504 making up the first composite image 508 ... a series of composite images 508 aggregating different numbers of individual image frames 504, and thus providing different levels of sensitivity, can be generated);
determining (30) a corresponding first stacked frame (Fs, Fs,g, Fs,b) on the basis of the subset of frames (Fsel) and as a function of the set of values of the shifting parameters (|(νM)|, θN) (Tchilian, ¶0010, the image data from many individual image frames can be accurately combined or stacked, enabling dim objects within the field of view of the multiple image frames to become visible; Tchilian, ¶0023, The aggregation of multiple image frames 504a-n collected at different times t0 to tn to form a composite or co-added frame 508 is illustrated in FIG. 5. As depicted in the figure, image data from one or more dim objects 120 may be present in at least some of the image frames 504, in the form of a signal from one or more pixels 210 within the respective image frames 504. In this example, image data corresponding to the first dim object 120a is present in all of the image frames 504a-n, while image data corresponding to the second dim object 120b is present in only two of the image frames 504c and 504d. Moreover, whether or not image data corresponding to a dim object 120 is present in some or all of the image frames 504, the strength of that signal may be insufficient for that signal to register as an object. For instance, it may be impossible to distinguish that signal from noise. Accordingly, it is desirable to aggregate the signals from the multiple frames 504a-n, in order to enable or facilitate the detection of the dim objects 120. Because the location of the dim objects 120, within each individual frame 504 varies as the platform 104 and/or the multiple mode star tracker 108 moves relative to the ECI, accurate co-addition and thus accurate detection of the objects 120 requires that the image frames 504a-n be aligned along a common boresight or reference axis 520. By registering the data from each image frame 504 in the same way relative to the ECI, signals collected from the same location in space by the different image frames 504 can be added to form the composite frame 508 ... By thus co-adding the frames of image data, the signals obtained from the same points in space can be accurately aligned, enabling enough signal to be accumulated to obtain an image of dim objects).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Tchilian and Shao because they are in the same field of endeavor. One skilled in the art would have been motivated to include the subsets of frames as taught by Tchilian in the system of Shao as an alternate means to accurately align frames to detect the location of dim objects in space (Tchilian, Abstract).
As per claim 2, Shao and Tchilian disclose the computer-implemented method for detecting objects in the field of view of the optical detection device (2) according to claim 1, wherein the objects are point-like objects (Shao, ¶0024, the at least one moving object includes an article from the group consisting of an asteroid and an unmanned aerial vehicle (UAV); Tchilian, ¶0017, The multiple mode star tracker 108 images a plurality of stars 112 within a field of view 116 of the multiple mode star tracker 108 ... the multiple mode star tracker 108 enables stacking of multiple image frames in order to significantly boost the signal-to-noise ratio (SNR) of the device, allowing the detection of dim objects 120, such as a distant star, or some other object, such as a space craft, space junk, a meteoroid).
As per claim 3, Shao and Tchilian disclose the computer-implemented method for detecting objects in the field of view of the optical detection device (2) according to claim 1, wherein said values indicating the trajectory comprise values of angular direction (|(νM)|, θN) (Shao, ¶0077, As can readily be appreciated, the location of moving objects in the images is not known a priori. Accordingly, the shift-and-add computations involve adopting a hypothesis that a given pixel within a frame images an object that is moving, and summing pixels across a sequence of preceding images corresponding to locations that would be occupied assuming that the object is moving at a given velocity. Each possible velocity (i.e. speed and direction) involves the generation of a different sum of pixel locations across the sequence of images; Shao, ¶0112, a thumbnail of the moving object may be generated and displayed or reported to a display device in real time. Some embodiments of the invention may also compute and report the object's size, position and/or velocity ... The thumbnail, in combination with other information such as velocity and trajectory, can also be utilized to classify the object as a certain type of object, or into a certain category. Certain embodiments of the invention generate an alert based on the classification of the moving object ... Examples of classifications include, but are not limited to, a UAV, asteroid, missile … and other categories of varying specificity).
As per claim 4, Shao and Tchilian disclose the computer-implemented method for detecting objects in the field of view of the optical detection device (2) according to claim 1, wherein the subset of frames (Fsel) comprising a respective first frame (F.sub.0) and a respective set of additional frames (Fsel′), and wherein the step of determining (30) the corresponding first stacked frame (Fs, Fs,g, Fs,b) on the basis of the subset of frames (Fsel) and as a function of each set of values of the shifting parameters (|(νM)|, θN) comprises, for each set of values of the shifting parameters (|(νM)|, θN): determining (30) a corresponding first set of values of shifting distances (dx, dy) as a function of the set of values of the shifting parameters (|(νM)|, θN) (Shao, ¶0007, Execution of the object tracking application directs the processor to receive a sequence of images, wherein at least one moving object is visible in relation to a background in the sequence of images; estimate pixel background values based on an average of pixel values within a sequence of images; subtract background pixel values from pixels in a sequence of images; compute sets of summed intensity values for different per frame pixel offsets from a sequence of images; Shao, ¶0013, the given per frame pixel offset is determined based on the fractional pixel shift and a velocity shift; Tchilian, ¶0008; Tchilian, ¶0010; Tchilian, ¶0017);
shifting (30) the pixels of each additional frame (Fsel′) as a function of the first set of values of the shifting distances (dx, dy) and of a time difference between the time instant of the first frame (F0) of the subset of frames (Fsel) and the time instant of the additional frame (Fsel′), to obtain a corresponding first set of shifted frame (Fsh) (Shao, ¶0070, an object tracking system applies a shift-and-add process to a sequence of short high-frame rate exposures of a scene, to synthetically render a long-exposure image of a moving object within the scene. Clustering processes can be utilized to determine the number of moving objects that are present based upon the content of the synthetically rendered long-exposure images. In many instances, a single moving object can appear as multiple high intensity pixels within such a synthetically rendered long-exposure image and a clustering process can be utilized to refine the location and/or velocity of the moving object); and
stacking (30) said corresponding first set of shifted frame (Fsh) with the first frame (F0) of the subset of frames (Fsel) to obtain the first stacked frame (Shao, ¶0076, Although the moving object 110 may appear faint or undetectable within a single frame, according to a number of embodiments of the invention, processing system 210 performs an object tracking process that can include a series of shift-and-add computations. These shift-and-add computations can be conceptually viewed as a processing in which the images within the sequence are shifted so that the moving object 110 within one image is matched up with the position of the moving object 110 within other images. Upon summing the pixels of the shifted images, the moving object 110 may appear more prominently in this synthetically rendered long-exposure image. A conceptual illustration of such shifted and added frames 500, with the object lined up at target track 510, is shown in FIG. 5).
As per claim 5, Shao and Tchilian disclose the computer-implemented method for detecting objects in the field of view of the optical detection device (2) according to claim 4, wherein the set of frames (F) comprises a respective first frame (F0) and a set of additional frames (FB) and wherein, for each selected set of values of shifting parameters (|(νg)|, θg), the step of determining (70) the corresponding second stacked frame (Fs,n, Fs,n,g, Fs,n,b) on the basis of the set of frames (F) and as a function of the selected set of values of shifting parameters (|(νg)|, θg) comprises, for each selected set of values of shifting parameters (|(νg)|, θg): determining (70) a corresponding second set of values of shifting distances (dx, dy) as a function of the selected set of values of shifting parameters (|(νg)|, θg) (Shao, ¶0032, The object tracking method further comprises estimating a velocity of the at least one moving object based on the per frame sub-pixel offset associated with the summed intensity value; and reporting the velocity of the at least one moving object to a display device in real time; Shao, ¶0098, the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined);
shifting (70) the pixels of each additional frame (FB) as a function of the second set of values of the shifting distances (dx, dy) and of a time difference between the time instant of the first frame (F0) of the set of frames (F) and the time instant of the additional frame (FB), to obtain a corresponding second set of shifted frame (Fsh) (Shao, ¶0098, the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined; Shao, ¶0099, summed intensity values from the sets of summed intensity values exceeding a threshold may be identified (410). A location of at least one moving object in the reference image may also be identified based on a summed intensity value from a set of summed intensity values exceeding a threshold); and
stacking (70) said corresponding second set of shifted frame (Fsh) with the first frame (F0) of the set of frames (F) to obtain the corresponding second stacked frame (Fs,n, Fs,n,g, Fs,n,b) (Shao, ¶0100, The high summed intensity values can thus include more than one peak, from which the moving object's position and velocity may be estimated, in accordance with some embodiments of the invention. In certain embodiments of the invention, the position and/or velocity can be refined at a sub-pixel level by determining a sub-pixel shift based on the pixel offsets associated with the plurality of peaks. The sub-pixel shift may be determined by weighting the peak pixel offsets, or by assuming per frame shifts with sub-pixel components. Interpolation can then be used to generate intensity values at the sub-pixel locations. These values can then be summed to generate a summed intensity value for the sub-pixel shift. The result should yield the highest intensity peak, with its value estimating a refined velocity of the moving object).
As per claim 6, Shao and Tchilian disclose the computer-implemented method for detecting objects in the field of view of the optical detection device (2) according to claim 1, wherein the step of determining (110) if the first stacked frame (Fs, Fs,g) classified as belonging to the first class (C1) and the second stacked frame (Fs,n, Fs,n,g, Fs,n,b) classified as belonging to the third class (C1) meet a requirement comprises: verifying if the maximum intensities of the first (Fs, Fs,g) and second stacked frames (Fs, Fs,g, Fs,b) classified as belonging, respectively, to the first and the third class (C1) are located in positions which fall within a confidentiality range (Shao, ¶0007, compute sets of summed intensity values for different per frame pixel offsets from a sequence of images, wherein a summed intensity value for a given per frame pixel offset is computed by summing intensity values of pixels in the images from the sequence of images determined using the given per frame pixel offset relative to a pixel location in a reference image from the sequence of images; identify summed intensity values from a set of summed intensity values exceeding a threshold; cluster identified summed intensity values exceeding the threshold corresponding to single moving objects; and identify a location of at least one moving object in an image based on at least one summed intensity value cluster; Shao, ¶0098, The result of performing shift-and-add calculations at each pixel location within a region of an image with respect to each of a number of different velocities is conceptually illustrated in FIG. 11. Each image in FIG. 11 corresponds to the summed intensities at a given pixel location, generated using a specific per frame shift relative to the pixel location to select pixels from a sequence of images, and then summing the selected pixels. The images correspond to different per frame integer pixel shifts in the x and/or y direction … the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined);
if said maximum intensities are located in positions which fall within a confidentiality range, identifying said maximum intensities as forming corresponding images of the object (Shao, ¶0007, compute sets of summed intensity values for different per frame pixel offsets from a sequence of images, wherein a summed intensity value for a given per frame pixel offset is computed by summing intensity values of pixels in the images from the sequence of images determined using the given per frame pixel offset relative to a pixel location in a reference image from the sequence of images; identify summed intensity values from a set of summed intensity values exceeding a threshold; cluster identified summed intensity values exceeding the threshold corresponding to single moving objects; and identify a location of at least one moving object in an image based on at least one summed intensity value cluster; Shao, ¶0098, The result of performing shift-and-add calculations at each pixel location within a region of an image with respect to each of a number of different velocities is conceptually illustrated in FIG. 11. Each image in FIG. 11 corresponds to the summed intensities at a given pixel location, generated using a specific per frame shift relative to the pixel location to select pixels from a sequence of images, and then summing the selected pixels. The images correspond to different per frame integer pixel shifts in the x and/or y direction … the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined; Shao, ¶0101, The example in FIG. 12 shows a per frame shift of (−1.3, 0) which is between the two highest peaks of (−1, 0) and (−2, 0). The selection of (−1.3, 0) could be based upon sampling a set of sub-pixel shifts within a range between of (−1, 0) to (−2, 0) or could be determined in a single calculation by determining a sub-pixel shift based upon a weighting of the intensity peaks at (−1, 0) and (−2, 0). A moving object can be determined to exist in the set of images when the absolute value (because the moving object could be darker or brighter than the background) of the highest peak exceeds a threshold value, or when a resampled thumbnail at the interpolated velocity exceeds a threshold value); and
determining that the pixels of the stacked frames (Fs, Fs,g) of the corresponding first set of stacked frames (Fs, Fs,g, Fs,b) classified as belonging to the first class (C1) and of pixels of stacked frames (Fs,n, Fs,n,g, Fs,n,b) of the corresponding second set of stacked frames (Fs,n, Fs,n,g, Fs,n,b) represent the image of the object (Shao, ¶0098, The result of performing shift-and-add calculations at each pixel location within a region of an image with respect to each of a number of different velocities is conceptually illustrated in FIG. 11. Each image in FIG. 11 corresponds to the summed intensities at a given pixel location, generated using a specific per frame shift relative to the pixel location to select pixels from a sequence of images, and then summing the selected pixels. The images correspond to different per frame integer pixel shifts in the x and/or y direction … the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined; Shao, ¶0101, The example in FIG. 12 shows a per frame shift of (−1.3, 0) which is between the two highest peaks of (−1, 0) and (−2, 0). The selection of (−1.3, 0) could be based upon sampling a set of sub-pixel shifts within a range between of (−1, 0) to (−2, 0) or could be determined in a single calculation by determining a sub-pixel shift based upon a weighting of the intensity peaks at (−1, 0) and (−2, 0). A moving object can be determined to exist in the set of images when the absolute value (because the moving object could be darker or brighter than the background) of the highest peak exceeds a threshold value, or when a resampled thumbnail at the interpolated velocity exceeds a threshold value).
As per claim 7, Shao and Tchilian disclose the computer-implemented method for detecting objects in the field of view of the optical detection device (2) according to claim 6, wherein the step of verifying if the maximum intensities of the first (Fs, Fs,g) and second stacked frames (Fs, Fs,g, Fs,b) classified as belonging, respectively, to the first and the third class (C1) are located in positions which fall within a confidentiality range comprises overlapping (110) the first stacked frame (Fs, Fs,g) classified as belonging to the first class (C1) with the second stacked frame (Fs,n, Fs,n,g, Fs,n,b) (Shao, ¶0019, clustering identified summed intensity values exceeding the threshold corresponding to single moving objects is performed by calculating a distance between a first summed intensity value and a second summed intensity value in four-dimensional space, the first and second summed intensity values being from identified summed intensity values exceeding the threshold; determining whether the first and second summed intensity values are neighbors based on the calculated distance; and when the first and second summed intensity values are determined to be neighbors, combining the first and second summed intensity values into a summed intensity value cluster; Shao, ¶0098, The result of performing shift-and-add calculations at each pixel location within a region of an image with respect to each of a number of different velocities is conceptually illustrated in FIG. 11. Each image in FIG. 11 corresponds to the summed intensities at a given pixel location, generated using a specific per frame shift relative to the pixel location to select pixels from a sequence of images, and then summing the selected pixels. The images correspond to different per frame integer pixel shifts in the x and/or y direction … the integer pixel shifts that yield the largest intensity peaks can be utilized to recompute intensity sums by interpolating pixels in the sequence of images corresponding to values at sub-pixel shifts. In this way, the accuracy of the estimated location of the moving object and/or its estimated velocity can be refined).
As per 8, Shao and Tchilian disclose a system for detecting objects in the field of view of an optical detection device (2) configured to carry out the method according to claim 7 (Shao, ¶0007, an object tracking system comprises a processor, a communications interface capable of transmitting a sequence of images to the processor, and a memory coupled with the processor and configured to store an object tracking application. Execution of the object tracking application directs the processor to receive a sequence of images).
As per claim 9, Shao and Tchilian disclose software loadable in a system (1) for detecting objects in the field of view of an optical detection device (2) and configured to allow, when run, the system to implement the method according claim 1 (Shao, ¶0079, Processing system 210 may perform processing locally, or it may partially compute information locally, with additional processing being performed on a set of GPUs that could be located locally or remotely. Processing system 210 includes a processor 212, which may refer to one or more devices within the computing device that can be configured to perform computations via machine readable instructions stored within a memory 220 of the processing system 210. The memory 220 may contain an object tracking application 222 that performs processes).
As per claim 10, Shao and Tchilian disclose computer-readable medium storing the software according to claim 9 (Shao, ¶0079, Processing system 210 may perform processing locally, or it may partially compute information locally, with additional processing being performed on a set of GPUs that could be located locally or remotely. Processing system 210 includes a processor 212, which may refer to one or more devices within the computing device that can be configured to perform computations via machine readable instructions stored within a memory 220 of the processing system 210. The memory 220 may contain an object tracking application 222 that performs processes).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Examiner, Art Unit 2668