Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11-17, 19-20, 22-23, 34-40 are rejected under 35 U.S.C. 103 as being unpatentable over Tohidypour et al. (Deep Learning Based Camera Switching for Sports Broadcasting, hereinafter referred to as Tohidypour) in view of Chapel et al. (Moving Objects Detection with a Moving Camera: A Comprehensive Review, hereinafter referred to as Chapel).
Regarding claim 11, Tohidypour teaches a computer-implemented method comprising: receiving first image data captured from a first perspective by a first physical camera and second image data captured from a second perspective by a second physical camera, wherein the first image data and the second image data indicate a sequence of object positions (met by sports broadcasting involves the use of several cameras located at different places; further met by processing live video feed). This is read in (2 Our Approach).
PNG
media_image1.png
231
666
media_image1.png
Greyscale
Tohidypour fails to teach predicting in real time a trajectory of the object in three-dimensional (3D) space; and for each of the first image data and the second image data: determining an apparent size of the object. However, Chapel amends this deficiency.
Chapel teaches tracking the trajectories of detected objects to construct a 3D subspace, enabling the 3D trajectory prediction of the claimed invention as well as updating the shape of trajectories. Further, Chapel teaches labeling clusters of points based on shape and size, enabling the ability to determine an apparent size of a detected object. This is read in (Page 41, Paragraph 1).
PNG
media_image2.png
88
620
media_image2.png
Greyscale
PNG
media_image3.png
131
614
media_image3.png
Greyscale
PNG
media_image3.png
131
614
media_image3.png
Greyscale
Therefore, it would have been prima facia obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to have modified Tohidypour to incorporate the teachings of Chapel in order to provide detection of moving objects in various challenging environments (Abstract).
Further, Chapel teaches determining a parallax effect (met by misaligned pixels correspond either to parallax or to a moving object) for the object based at least in part on an analysis of respective image data compared with updated image data computed by a simulated change in angle of a respective physical camera; This is read in (Page 46, Paragraph 2).
PNG
media_image4.png
267
667
media_image4.png
Greyscale
See also (Page 28, Paragraph 1) for rotation angles of new frames used with stored information to obtain first coarse registration refined by estimation of transformation parameters between new frame and rough mosaic region. This further meets using updated image using simulated change in perspective along with compared with respective image data.
PNG
media_image5.png
441
664
media_image5.png
Greyscale
Determining a viewable portion of the trajectory of the object is met by the combined prior art demonstrating the ability to compute trajectories in a 3-d space based on multiple image data as previously read (Chapel, Page 41, Paragraph 1).
Tohidypour teaches selecting, as a selected camera, one of the first physical camera or the second physical camera based at least in part on the determined apparent size of the object, the determined motion parallax effect for the object, or the determined viewable portion of the trajectory of the object (met by camera switching scheme; further met by prior art’s ability to determine size and trajectory of an object as well as parallax from video feed); and receiving, using the selected camera, new image data of at least a portion of the trajectory of the object (met implicitly by the prior art’s ability to use previous and currently received frames to compute trajectory). This is read in (2.3 Camera Switching).
PNG
media_image6.png
122
669
media_image6.png
Greyscale
PNG
media_image6.png
122
669
media_image6.png
Greyscale
PNG
media_image7.png
114
669
media_image7.png
Greyscale
Regarding claim 12, Chapel as read at (Page 28, Paragraph 1) in the rejection of claim 11, incorporated herein, meets determining at least one intermediate perspective between a perspective of the selected camera and an original camera angle (met by rotation angles of new frames used with stored information to obtain first coarse registration refined by estimation of transformation parameters between new frame and rough mosaic region).
Chapel further teaches receiving intermediate camera image data of the object from the at least one intermediate perspective before the receiving the new image data of the object from the selected camera (met by constructing a panoramic background model). This is read in (Figure 6; Page 25, Paragraph 1).
PNG
media_image8.png
699
832
media_image8.png
Greyscale
Regarding claim 13, Tohidypour in view of Chapel as read in the rejection of claims 11 and 12, incorporated herein, meets wherein the trajectory of the object in the three- dimensional space is predicted based at least in part on data received from one more sensors at the object (met by several cameras located at different places; Tohidypour, 2 Our Approach).
Regarding claim 14, Tohidypour in view of Chapel as read in the rejection of claims 11 and 12, incorporated herein, meets wherein the trajectory of the object in the three- dimensional space is predicted based at least in part on the first image data or the second image data (met by the trajectories from two consecutive frames are used; Chapel, Page 41, Paragraph 1).
Regarding claim 15, Tohidypour in view of Chapel as read in the rejection of claims 11 and 12, incorporated herein, meets wherein the viewable portion of the trajectory of the object is determined with respect to an entirety of the trajectory of the object (met implicitly by determining the trajectory of an object, thereby allowing the viewable portion of the trajectory to be determined with respect to the entire trajectory).
Regarding claim 16, Tohidypour in view of Chapel as read in the rejection of claims 11 and 12, incorporated herein, meets wherein the apparent size of the object is determined based at least in part on a pixel count in one or more video frames (met by label the cluster of trajectories
based on the shape and the size, Chapel, Page 41, Paragraph 1). The clustering of trajectories would inherently be determined by the pixels of the image, and therefore a pixel count is an innate function of the prior art necessitated by this process.
Regarding claim 17, performing the previously claimed functions for two cameras at two distances and selecting the camera with the greatest objective value is met by Tohidypour in view of Chapel as read in the rejection of claim 11, incorporated herein, read in (2.3 Camera Switching), which teaches the algorithm considers the position and confidence level of all the detected objects to assign a score for each camera view. The other features of the claim are limitations previously addressed in the rejection of claims 11-16, incorporated herein.
See also (Tohidypour, Figure 3.).
PNG
media_image9.png
684
509
media_image9.png
Greyscale
Regarding claim 19, Chapel teaches repositioning the selected camera according to the determined camera perspective (met by kinematic calibration step based on epipolar geometry is used to rotate the PTZ camera). This is read in (Page 31, Paragraph 1). Additional limitations of the claim are met by the rejection of claims 11-17, incorporated herein.
PNG
media_image10.png
543
666
media_image10.png
Greyscale
Regarding claim 20, Tohidypour in view of Chapel as read in (Chapel, Page 41, Paragraph 1) in the rejection of claims 11 and 12, incorporated herein, meets wherein the trajectory of the object is predicted by computing at least two coordinates in 3D space indicating successive positions of the object (met by using three long term trajectories to construct a 3D subspace), and determining a time interval between the at least two coordinates (met by trajectories from two consecutive frames are used rather than long term trajectories to reduce computation time and memory resource, thus enabling the time interval to be derived from the frame rate of a camera should one of ordinary skill in the art have need to do so).
Regarding claim 22, the claim is substantially identical to claim 11, the analysis of which is incorporated herein.
Regarding claim 23, the claim is substantially identical to claim 12, the analysis of which is incorporated herein.
Regarding claim 34, the claim the claim is substantially identical to claim 11, the analysis of which is incorporated herein.
Regarding claim 35, the claim is substantially identical to claim 12, the analysis of which is incorporated herein.
Regarding claim 36, the claim is substantially identical to claim 13, the analysis of which is incorporated herein.
Regarding claim 37, the claim is substantially identical to claim 14, the analysis of which is incorporated herein.
Regarding claim 38, the claim is substantially identical to claim 15, the analysis of which is incorporated herein.
Regarding claim 39, the claim is substantially identical to claim 16, the analysis of which is incorporated herein.
Regarding claim 40, the claim is substantially identical to claim 17, the analysis of which is incorporated herein.
Allowable Subject Matter
Claims 18, 21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 18 recites the viewable portion of the trajectory of the object is determined based at least in part on a visibility of the trajectory of the object in a field of view from a camera perspective and based at least in part on one or more obstructions determined in the field of view from the camera perspective. These limitations are not met by the prior art.
Claim 21 recites inserting a graphic in the video data underneath the object indicating on the surface a location corresponding to the position of the object. This limitation is not met by the prior art.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW JAMES BODNARK whose telephone number is (703)756-5378. The examiner can normally be reached 8a-5p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW JAMES BODNARK/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights