Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 5 and 6 are objected to because of the following informalities: recites “Ordered Surface matrices…”. Ordered Surface should not be capitalized. Appropriate correction is required.
Claims 4 and 13 are objected to because of the following informalities: recites “the maximum number of the recorded events…” in line 1. Since the limitation is not recited in previous related claim limitations, Claims 4 and 13 should be amended to recite “a maximum number of the recorded events”. Appropriate correction is required.
Claim 6 is objected to because of the following informalities: recites “… employing a Harris detector to the patch to generate a Harris score;” in line 3. For continuity as recited in the previous limitations, Claim 6 should be amended to recite “…employing a Harris detector to the patch of elements to generate a Harris score;” . Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by luvHarris: A Practical Corner Detector for Event-Cameras to Glover et al., hereinafter, “Glover”.
Claim 1. Glover teaches A method for adaptive corner detection in machine vision, comprising: [Abstract] The result is a practical, real-time, and robust corner detector…when using a high-resolution event-camera in real-time
[Introduction] CORNER detection is used for motion estimation and feature point identification among other machine vision tasks [1]…we present yet another method for performing corner detection with event-cameras
obtaining one or more event data from a dynamic vision sensor; [3.2 As-Fast-as-Possible Computation: Harris Look-Up] A key concept of luvHarris is that the asynchronous event-pipeline… A single instance of L is used for all incoming events
[3.3 Throughput Limitations] Real-time processing is achieved despite a variable number of input events,…
[Introduction] we present yet another method for performing corner detection with event-cameras. Examiner understand event-cameras to be dynamic vision sensors
capturing and arranging, by a machine vision processor, a plurality of recorded events of the event data into a 2D array; [3.2 As-Fast-as-Possible Computation: Harris Look-Up] The output of cv::cornerHarris is a 2D array which is directly used to populate the 2D look-up-table, L, such that the value at each entry is the Harris score.
[3.3 Throughput Limitations] Real-time processing is achieved despite a variable number of input events, as the processing for each event (TOS) is decoupled from heavy algorithm processing (Harris). – paragraphs 3 and 4, Examiner interprets 3.3 to be the machine vision processor.
transforming, by the machine vision processor the 2D array into one or a plurality of Ordered Surface matrices by populating one or a plurality of empty Image Matrices with the recorded events based on their coordinates and assigning order values; [2 Background] The algorithm was event-by-event in that for each incoming event, the surface was incrementally updated, and the Harris response was computed only locally around the position on the surface at which the event occurred. The Harris response is dependent on the Eigenvalues of the image derivative, in a square patch around the event position:…
[Fig. 3] (a) is produced with kTOS = 3 leading to TTOS =241, below which all values are set to 0. The brightness of each pixel represents the value in the TOS. (b) and (c) show that strong edges and corners are present in the visual signal, while blank regions are either zero (black) Examiner interprets Fig. 3(a) to be the ordered surface matrix and the zero’s to be the empty image matrix.
applying, by the machine vision processor, a corner detector to the Order Surface matrices; [3.1 Event-by-Event Computation: Threshold-Ordinal Surface]
for any input event vi ¼ hx; y; ti, the full processing to update the TOS and assign the corner classification vc defined in Algorithm 1…The appropriate selection of TTOS for corner detection is made to form an edge of 2 pixels thick and TTOS =2 (2kTOS + 1). For a typical patch size of 7 x 7 (kTOS = 3), TTOS will take the value of 14, which corresponds to a line of 7 pixels, 2 pixels thick, for a perfect clean edge passing through the region., Fig. 3 (c)
[2.2 Considerations for Applying the Harris Algorithm to Event-Data] Original Harris corner detection is applied over an image, in which pixel values are typically bound between 0 and 255…
and outputting a corner detection result from the corner detector. [3.1 Event-by-Event Computation: Threshold-Ordinal Surface] …The appropriate selection of TTOS for corner detection
[3.2 As-Fast-as-Possible Computation: Harris Look-Up] The output of cv::cornerHarris is a 2D array which is directly used to populate the 2D look-up-table, L, such that the value at each entry is the Harris score.
Claim 2. Glover further teaches wherein the recorded events are arranged in global temporal ordering in the 2D array. [3.1 Event-by-Event Computation: Threshold-Ordinal Surface] The TOS is somewhat similar to the Speed Invariant Time Surface [8] (SITS), in which it was concluded that the ordinal method (the surface value corresponds to the order of event arrival)
[3.2 As-Fast-as-Possible Computation: Harris Look-Up] The output of cv::cornerHarris is a 2D array which is directly used to populate the 2D look-up-table, L, such that the value at each entry is the Harris score… the time in which L was generated, Lt
Claim 3. Glover further teaches wherein the recorded events are arranged in a plurality of rows in the 2D array, [3.2 As-Fast-as-Possible Computation: Harris Look-Up] The output of cv::cornerHarris is a 2D array which is directly used to populate the 2D look-up-table Examiner understands a 2D array to be rows and columns.
and each of the rows has a series of the recorded events arranged in global temporal ordering; [Fig. 3]
and wherein the numbers of the recorded events in the rows are the same, and the rows are arranged in global temporal ordering. [3.1 Event-by-Event Computation: Threshold-Ordinal Surface] As the TOS value is set to 255 for new events, and new neighbouring events subtract 1 from the entire region, once a TOS values reaches 255 - TTOS setting it to 0 achieves this desired number. Examiner interprets TOS value is set to 255 for new events and new neighbouring events subtract 1 to be the same
Claim 4. Glover further teaches wherein the maximum number of the recorded events in every row ranges from 25 to 100. [3.3 Throughput Limitations] Real-time processing is achieved despite a variable number of input events, as the processing for each event (TOS) is decoupled from heavy algorithm processing (Harris)… V is the total number of events…
Claim 5. Glover further teaches wherein each Ordered Surface matrix is derived from all the rows of the recorded events in the 2D array. [2 Background] The Harris response is dependent on the Eigenvalues of the image derivative, in a square patch around the event position:… The Harris response is related to the Eigenvalues of the matrix M
[3.2 As-Fast-as-Possible Computation: Harris Look-Up] The output of cv::cornerHarris is a 2D array which is directly used to populate the 2D look-up-table, L,
Claim 6. Glover further teaches wherein the step of applying the corner detector to one of the Ordered Surface matrices includes: extracting a patch of elements in the Ordered Surface matrix; [3 Look-up Event-Harris] Algorithm 1: an asynchronous, event-by-event update of a threshold-ordinal surface (TOS),
[3.1 Event-by-Event Computation: Threshold-Ordinal Surface] The TOS, visualised in Fig. 3, provides a coherent and bound spatial representation of the asynchronous events, partially maintaining the information about their temporal order
employing a Harris detector to the patch to generate a Harris score; [3 Look-up Event-Harris] Algorithm 2: the calculation of a Harris-score lookup-table [3.1 Event-by-Event Computation: Threshold-Ordinal Surface]
and comparing the Harris score generated by the Harris detector with a predefined threshold. [3.1 Event-by-Event Computation: Threshold-Ordinal Surface] TR is the Harris score threshold
[2 Background] An event is classified as a corner-event if the Harris response is above a threshold TR
[4.2 Corner Accuracy] The Harris algorithm is a widely accepted baseline in the field of computer vision. Finally, a threshold was applied to the scores to result in the set of true corner events…
Claim 7. Glover further teaches further comprising: performing sort normalization to the patch of elements before employing the Harris detector. [3.1 Event-by-Event Computation: Threshold-Ordinal Surface] The TOS, visualised in Fig. 3… As the TOS value is set to 255 for new events, and new neighbouring events subtract 1 from the entire region, once a TOS values reaches 255 - TTOS setting it to 0 achieves this desired number. Examiner interprets the above passage to be sort normalization as it is described in the present application specification [0110]…Subsequently, these values undergo sort normalization, where the normalized pixel value is computed as 255 minus the index.
Claim 8. Glover further teaches wherein the patch has M rows and N columns, and the M ranges from 7 to 11, and the N ranges from 7 to 11. [Fig. 3] (a)
Claim 9. Glover further teaches further comprising: applying a spatial-temporal correlation filter to the event data before organizing the recorded events into the 2D array. [3.1 Event-by-Event Computation: Threshold-Ordinal Surface] The TOS, visualised in Fig. 3, provides a coherent and bound spatial representation of the asynchronous events, partially maintaining the information about their temporal and attempts to capture the most up-to-date position of edges in the scene.
Claim 10. Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim 11. Reviewed and analyzed in the same way as claim 2. See the above analysis and rationale.
Claim 12. Reviewed and analyzed in the same way as claim 3. See the above analysis and rationale.
Claim 13. Reviewed and analyzed in the same way as claim 4. See the above analysis and rationale.
Claim 14. Reviewed and analyzed in the same way as claim 5. See the above analysis and rationale.
Claim 15. Reviewed and analyzed in the same way as claim 6. See the above analysis and rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DELOMIA L GILLIARD whose telephone number is (571)272-1681. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DELOMIA L GILLIARD/Primary Examiner, Art Unit 2661