DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number DE10 2023 211 846.7 dated 11/28/2023.
Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS(s) dated 11/25/2024 has/have been considered and placed in the application file.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-7 and 9-11 is/are rejected under 35 U.S.C. 103 as obvious over Merkel et al (US 20100103266 A1, hereafter referred to as Merkel) in view of Huang, Shiyao et al (Huang, S., Ying, X., Rong, J., Shang, Z., & Zha, H. (2016). Camera calibration from periodic motion of a pedestrian. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3025-3033)., hereafter referred to as Huang).
Claim 1
Regarding Claim 1, Merkel teaches A computer-implemented method for determining extrinsic camera parameters of a camera (1), the method comprising:
capturing image data from a camera (Merkel in ¶23-25, 41 discloses receiving surveliance pictures/video sequences captured by surveillance cameras),
determining, via a computer, foot strike points (11 a, b) of at least one object (8) walking on a monitoring base area (2) based on image data of the camera (1) (Merkel in ¶10, 14 discloses image based tracking of the ground contact base of a pedestrian);
wherein the step sections (12 a, b) each have a section length on the monitoring base area (2) (Merkel in ¶11, 15-17, 38-39 discloses converting seperations between tracked ground-contact positions into real world ground plane distances), and
determining, via the computer, a pitch angle (3) and/or a roll angle (4) of the camera (1) relative to the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2) (Merkel in ¶9, 17, 20, 35-39 discloses determining camera inclincation and roll from the image locations of trajectory points and corresponding ground plane distances)
Merkel does not explicitly teach all of determining, via a computer, foot strike points (11 a, b) of at least one object (8)
However, Huang teaches determining, via a computer, foot strike points (11 a, b) of at least one object (8) walking on a monitoring base area (2) based on image data of the camera (1) (Huang in pp. 3026-3028, §§1-3, Fig. 1 discloses detecting toe positions from pedestrian image blobs in frames where the front shoe has just touched the ground and remains stationary),
wherein the foot strike points (11 a, b) form at least three step sections (12 a, b), wherein each step section (12 a, b) is defined by two foot strike points (11 a, b) (Huang in p. 3026 discloses that the minimum calibration data are three continous steps/four continuous shoe prints on the ground; pp. 3027, 3031, §§2, 5.2 discloses organizing left and right toe positions into repreated ground-pllane point series. Straight segments joining successive contact points or successive same shoe points are respectively single step or double step sections, each defined by two detected points), and
wherein the step sections (12 a, b) each have a section length on the monitoring base area (2) (Huang in pp. 3026-3027 discloses that adjacent same shoe toe positions on the greound plane are equidistant and uses their projective metric relationship), and
determining, via the computer, a pitch angle (3) and/or a roll angle (4) of the camera (1) relative to the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2) (Huang in pp. 3026-3031 discloses using the detected shoe contact locations and their equal spacing relationship to recover orthogonal ground/world vanishing points and then the camera rotation matrix).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Merkel by incorporating the ground contact shoe position extraction, at least three step sections, and equal length relationship that is taught by Huang, since both references are analogous art in the field of image based extrinsic calibration of cameras; thus, one of ordinary skilled in the art would be motivated to combine the references since Merkel’s trajectory distance calibration with Huang’s repeated shoe contact geometry yields the predictable result of determining camera inclination and roll from the locations and relative lengths of pedestrian step sections, thereby providing robust calibration without a dedicated scene calibration object.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 2
Regarding Claim 2, Merkel in view of Huang teaches A method according to claim 1, wherein the at least three step sections (12 a, b) each define a straight line section on the monitoring base area (2), wherein the straight line sections intersect (Merkel in ¶16 discloses that the trajectory between two position data points is straight or approximately straight; Huang in pp. 3026, 3031 discloses three continuous steps formed by four continuous ground contact shoe positions. Each two point step is a straight segment, and consecutive segments share a shoe contact endpoint).
Claim 3
Regarding Claim 3, Merkel in view of Huang teaches A method according to claim 1, wherein the three step sections (12 a, b) comprise at least one common foot strike point (11 a, b) and/or are associated with a common object (8) (Huang in pp. 3026-3027, 3031 discloses using three continuous steps of the same pedestrian. Consecutive step sections share a ground contact shoe point and are all associated with the same pedestrian).
Claim 4
Regarding Claim 4, Merkel in view of Huang teaches A method according to claim 1, wherein the step sections (12 a, b) are not connected and/or are associated with different objects (8) and/or different trajectories (13) of the same object (8) and/or spaced trajectory sections of the trajectory (13) of the same object (8) (Merkel in ¶18 discloses using and statistically combining a plurality of trajectories from a moving object and/or trajectories from various moving objects; ¶40 discloses collecting clustered trajectories over a long observation period).
Claim 5
Regarding Claim 5, Merkel in view of Huang teaches A method according to claim 1, wherein the step sections (12 a, b) each form a step of the object (8) or a double step of the object (8) (Huang in p. 3026 discloses three continuous steps/four continuous shoe prints; pp. 3027, 3031 discloses separately grouping left and right shoe contacts so a section between successive contacts of the same shoe is a double step/stride).
Claim 6
Regarding Claim 6, Merkel in view of Huang teaches A method according to claim 1, wherein a height (5) of the camera (1) relative to the monitoring base area (2) is determined based on at least a section length of one of the step sections (12 a, b) (Merkel in ¶20 discloses estimating camera height from the ascertained real-world trajectory distances and the picture to world transformation. In the combined method, those real-world distances are Huang’s foot contact step section distances, so the height is determined based on at least one such section length).
Claim 7
Regarding Claim 7, Merkel in view of Huang teaches A method according to claim 1, wherein the absolute section length is estimated as a function of the time needed to traverse the step section (12 a, b) (Merkel in ¶15 discloses calculating a real-world distance by multiplying an assumed pedestrian speed by the time interval between image positions; ¶16 discloses substituting the actual temporal separation when the positions are not time equidistant. Applied to Huang’s successive contact positions, this estimates the absolute step section length as a function of traversal time).
Claim 9
Regarding Claim 9, Merkel teaches An evaluation module (7) for determining extrinsic camera parameters of a camera (1) having an interface for connection to a data source via data technology for receiving image data, wherein the evaluation module (7) is configured to
determine foot strike points (11 a, b) of at least one object (8) walking on a monitoring base area (2) based on image data (Merkel in ¶10, 14 discloses image based tracking of the ground contact base of a pedestrian);
wherein the step sections (12 a, b) each have a section length on the monitoring base area (2) (Merkel in ¶11, 15-17, 38-39 discloses converting seperations between tracked ground-contact positions into real world ground plane distances), and
determine a pitch angle (3) and/or a roll angle (4) of the camera (1) relative to the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2) (Merkel in ¶9, 17, 20, 35-39 discloses determining camera inclincation and roll from the image locations of trajectory points and corresponding ground plane distances)
Merkel does not explicitly teach all of determine foot strike points (11 a, b) of at least one object (8)
However, Huang teaches determine foot strike points (11 a, b) of at least one object (8) walking on a monitoring base area (2) based on image data of the camera (1) (Huang in pp. 3026-3028, §§1-3, Fig. 1 discloses detecting toe positions from pedestrian image blobs in frames where the front shoe has just touched the ground and remains stationary),
wherein the foot strike points (11 a, b) form at least three step sections (12 a, b), wherein each step section (12 a, b) is defined by two foot strike points (11 a, b) (Huang in p. 3026 discloses that the minimum calibration data are three continous steps/four continuous shoe prints on the ground; pp. 3027, 3031, §§2, 5.2 discloses organizing left and right toe positions into repreated ground-pllane point series. Straight segments joining successive contact points or successive same shoe points are respectively single step or double step sections, each defined by two detected points), and
wherein the step sections (12 a, b) each have a section length on the monitoring base area (2) (Huang in pp. 3026-3027 discloses that adjacent same shoe toe positions on the greound plane are equidistant and uses their projective metric relationship), and
determine a pitch angle (3) and/or a roll angle (4) of the camera (1) relative to the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2) (Huang in pp. 3026-3031 discloses using the detected shoe contact locations and their equal spacing relationship to recover orthogonal ground/world vanishing points and then the camera rotation matrix).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Merkel by incorporating the ground contact shoe position extraction, at least three step sections, and equal length relationship that is taught by Huang, since both references are analogous art in the field of image based extrinsic calibration of cameras; thus, one of ordinary skilled in the art would be motivated to combine the references since Merkel’s trajectory distance calibration with Huang’s repeated shoe contact geometry yields the predictable result of determining camera inclination and roll from the locations and relative lengths of pedestrian step sections, thereby providing robust calibration without a dedicated scene calibration object.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 10
Regarding Claim 10, Merkel in view of Huang teaches A camera (1) comprising the evaluation module (7) according to claim 9 (Merkel in ¶23-25 discloses an input module for receiving camera images, an object tracking module for determining a trajectory from the images, and a calibration module for determining the camera calibration).
Claim 11
Regarding Claim 11, Merkel teaches A non-transitory, computer-readable medium containing instructions that when executed by a computer cause the computer to
determine foot strike points (11 a, b) of at least one object (8) walking on a monitoring base area (2) based on image data (Merkel in ¶10, 14 discloses image based tracking of the ground contact base of a pedestrian);
wherein the step sections (12 a, b) each have a section length on the monitoring base area (2) (Merkel in ¶11, 15-17, 38-39 discloses converting seperations between tracked ground-contact positions into real world ground plane distances), and
determine a pitch angle (3) and/or a roll angle (4) of the camera (1) relative to the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2) (Merkel in ¶9, 17, 20, 35-39 discloses determining camera inclincation and roll from the image locations of trajectory points and corresponding ground plane distances)
Merkel does not explicitly teach all of determine foot strike points (11 a, b) of at least one object (8) the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2).
However, Huang teaches determine foot strike points (11 a, b) of at least one object (8) walking on a monitoring base area (2) based on image data of the camera (1) (Huang in pp. 3026-3028, §§1-3, Fig. 1 discloses detecting toe positions from pedestrian image blobs in frames where the front shoe has just touched the ground and remains stationary),
wherein the foot strike points (11 a, b) form at least three step sections (12 a, b), wherein each step section (12 a, b) is defined by two foot strike points (11 a, b) (Huang in p. 3026 discloses that the minimum calibration data are three continous steps/four continuous shoe prints on the ground; pp. 3027, 3031, §§2, 5.2 discloses organizing left and right toe positions into repreated ground-pllane point series. Straight segments joining successive contact points or successive same shoe points are respectively single step or double step sections, each defined by two detected points), and
wherein the step sections (12 a, b) each have a section length on the monitoring base area (2) (Huang in pp. 3026-3027 discloses that adjacent same shoe toe positions on the greound plane are equidistant and uses their projective metric relationship), and
determine a pitch angle (3) and/or a roll angle (4) of the camera (1) relative to the monitoring base area (2) based on the location of the step sections (12 a, b) as well as a ratio of the section lengths on the monitoring base area (2) and/or based on the absolute section lengths on the monitoring base area (2) (Huang in pp. 3026-3031 discloses using the detected shoe contact locations and their equal spacing relationship to recover orthogonal ground/world vanishing points and then the camera rotation matrix).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Merkel by incorporating the ground contact shoe position extraction, at least three step sections, and equal length relationship that is taught by Huang, since both references are analogous art in the field of image based extrinsic calibration of cameras; thus, one of ordinary skilled in the art would be motivated to combine the references since Merkel’s trajectory distance calibration with Huang’s repeated shoe contact geometry yields the predictable result of determining camera inclination and roll from the locations and relative lengths of pedestrian step sections, thereby providing robust calibration without a dedicated scene calibration object.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as obvious over Merkel et al (US 20100103266 A1, hereafter referred to as Merkel) and Huang, Shiyao et al (Huang, S., Ying, X., Rong, J., Shang, Z., & Zha, H. (2016). Camera calibration from periodic motion of a pedestrian. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3025-3033)., hereafter referred to as Huang), further in view of Sun et al (Sun, Y., Hare, J. S., & Nixon, M. S. (2016, November). Detecting acceleration for gait and crime scene analysis. In 7th International Conference on Imaging for Crime Detection and Prevention (ICDP 2016) (pp. 1-6). IET., hereafter referred to as Sun).
Claim 8
Regarding Claim 8, Merkel in view of Huang teaches A method according to claim 1.
Merkel in view of Huang does not explicitly teach all of wherein the foot strike points (11 a, b) are determined by digital image processing via the analysis of the optical flow in the image data.
However, Sun teaches wherein the foot strike points (11 a, b) are determined by digital image processing via the analysis of the optical flow in the image data (Sun in p. 1, Abstract, §1 discloses that heel strike frames and positions are estimated from radial acceleration derived from optical flow using DeepFlow. pp. 2-3, §§2-3 discloses computing acceleration by differencing consecutive optical flow estimates, identifying the frame in which radial acceleration of the leading foot peaks, and locating the heel strike at the center of radial acceleration).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Merkel in view of Huang by incorporating the optical flow derived acceleration analysis for locating heel strikes that is taught by Sun, since both references are analogous art in the field of image analysis of pedestrian gait; thus, one of ordinary skilled in the art would be motivated to combine the references since Merkel in view of Huang’s pedestrian based camera calibration method with Sun’s optical flow heel strike detector yields the predictable result of supplying the calibration with automatically located foot ground contact points from consecutive video frames, thereby improving contact point localization without manual annotation.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN P CASCAIS whose telephone number is (703) 756-5576. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mr. O'Neal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.P.C./Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Date: 8/20/2026
Prosecution Insights