DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/23/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Hanna et al. US PG-Pub(US 20100013917 A1) in view of Choi et al. US PG-Pub(US 20250037297 A1).
Regarding Claim 1, Hanna teaches an electronic system (Fig. 1) comprising: a camera device including a camera and a motor configured to rotate the camera in at least one of a pan rotation direction and a tilt rotation direction(¶[0024], “the system 100 comprises a camera pan/tilt/zoom (PTZ) module 102 that controls the pan/tilt/zoom parameters of at least one imaging sensor 104 (e.g., a visible or infrared camera),” discloses a camera system that controls the pan/tilt/zoom parameters of the imaging sensor.); a first server device configured to provide the motor with an angle control signal for obtaining first image data corresponding to a first scene including a target object by the camera (¶[0025],”The camera/PTZ module 102 is coupled to one or more imaging sensors such as, for example, cameras 104 (as shown in FIGS. 1 and 10) that are capable of capturing and transmitting video signals to the system 100 generally (but not exclusively) in an NTSC signal format. For example, the camera 104 can be a visible light camera transmitting video signals at a rate of approximately 30 Hz in either a 720.times.488 progressive scan or a 720.times.244 interlaced format.”, ¶[0025] discloses transmitting the video signals to a remote system.); wherein the camera device is configured to: rotate the camera in the pan rotation direction by a first angle and in the tilt rotation direction by a second angle based on the angle control signal(¶[0026], “the detection/PTZ control module 108 sends commands such that the camera(s) 104 sweep across the surveillance area. As the camera(s) 104 point further into the distance of such area, the detection/PTZ control module 108 can optionally send commands to zoom in on a particular object. Such commands may be manual on the part of a system operator or a guard, or automatically produced in response to an object being detected in the field of view of the camera. The camera/PTZ module 102 provides a series of coordinates that the system 100 recognizes as particular camera position for a given video signal. Thus, it is possible to map the camera position in the real world (pan, tilt, zoom parameters that are herein defines as PTZ coordinates) to the captured images (image or pixel coordinates).”, ¶[0026] discloses controlling the PTZ camera such that it can detect the object in the field of view.); obtain the first image data by the rotated camera(¶[0033], “The method 200 proceeds to step 214 where the next image representing a fixed location is captured, processed and mapped according to steps 204 through 212 as described above. When all of the images constituting the area under surveillance are so processed, the Zone Map is complete and the method ends at step 216.”, ¶[0033] discloses a next image is captured when the camera is adjusted.); and provide the first server device with the first image data and angle information indicating a rotation state of the camera(¶[0026], “The camera/PTZ module 102 provides a series of coordinates that the system 100 recognizes as particular camera position for a given video signal. Thus, it is possible to map the camera position in the real world (pan, tilt, zoom parameters that are herein defines as PTZ coordinates) to the captured images (image or pixel coordinates)”, discloses coordinates and position of the camera are transmitted to the server.)
Hanna does not explicitly teach a second server device, and wherein the second server device is configured to: determine estimated candidate coordinates of coordinates, at which the target object is located, in three-dimensional (3D) space based on 3D map information and the angle information received from the first server device.
Choi teaches a second server device(Fig. 6, 630 – shows a first server and 640 shows a second server.), and wherein the second server device is configured to: determine estimated candidate coordinates of coordinates, at which the target object is located, in three-dimensional (3D) space based on 3D map information and the angle information received from the first server device. (¶[0039], “the set of attributes of the at least one background object of the scene can include location information describing at least one location of the at least one background object represented by the 2D image data, and/or at least one measure of distortion of the at least one background object based on a 2D projection of the scene. 2D projection refers to the process of mapping points from 3D space onto respective points of a 2D plane, which can be used to transform the 3D scene into the 2D image data captured by the camera 120.”, ¶[0039] discloses estimating 3d coordinates of a target object using the 2d image acquired and ¶[0043] “In some embodiments, the output of the camera pose identification model is used to as an input to generate a 3D representation of the subject depicted in the 2D image data. In some embodiments, the engine 140 can analyze at least one movement of the subject by using the 3D representation (e.g., evaluating movement and changes in movement).”, discloses generating a 3d representation using the image acquired from the camera system.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Hanna with Choi in order to using 3d coordinates to determine the location of the object. One skilled in the art would have been motivated to modify Hanna in this manner in order to combine data channels to determine camera pose. (Choi, ¶[0002])
Regarding Claim 9, Hanna teaches a method of operating an electronic system including a camera device(¶[0024], “the system 100 comprises a camera pan/tilt/zoom (PTZ) module 102 that controls the pan/tilt/zoom parameters of at least one imaging sensor 104 (e.g., a visible or infrared camera),” discloses a camera system that controls the pan/tilt/zoom parameters of the imaging sensor.), a first server device( ¶[0025] discloses transmitting video signals to a server.), the method comprising: providing, by the first server device, the camera device with an angle control signal for obtaining first image data corresponding to a first scene including a target object (¶[0025],”The camera/PTZ module 102 is coupled to one or more imaging sensors such as, for example, cameras 104 (as shown in FIGS. 1 and 10) that are capable of capturing and transmitting video signals to the system 100 generally (but not exclusively) in an NTSC signal format. For example, the camera 104 can be a visible light camera transmitting video signals at a rate of approximately 30 Hz in either a 720.times.488 progressive scan or a 720.times.244 interlaced format.”, ¶[0025] discloses transmitting the video signals to a remote system.); rotating, by the camera device, a camera of the camera device in a pan rotation direction by a first angle and in a tilt rotation direction by a second angle based on the angle control signal;(¶[0026], “the detection/PTZ control module 108 sends commands such that the camera(s) 104 sweep across the surveillance area. As the camera(s) 104 point further into the distance of such area, the detection/PTZ control module 108 can optionally send commands to zoom in on a particular object. Such commands may be manual on the part of a system operator or a guard, or automatically produced in response to an object being detected in the field of view of the camera. The camera/PTZ module 102 provides a series of coordinates that the system 100 recognizes as particular camera position for a given video signal. Thus, it is possible to map the camera position in the real world (pan, tilt, zoom parameters that are herein defines as PTZ coordinates) to the captured images (image or pixel coordinates).”, ¶[0026] discloses controlling the PTZ camera such that it can detect the object in the field of view.); obtaining, by the camera device, the first image data; (¶[0033], “The method 200 proceeds to step 214 where the next image representing a fixed location is captured, processed and mapped according to steps 204 through 212 as described above. When all of the images constituting the area under surveillance are so processed, the Zone Map is complete and the method ends at step 216.”, ¶[0033] discloses a next image is captured when the camera is adjusted.); providing, by the camera device, the first server device with the first image data and angle information indicating a rotation state of the camera (¶[0026], “The camera/PTZ module 102 provides a series of coordinates that the system 100 recognizes as particular camera position for a given video signal. Thus, it is possible to map the camera position in the real world (pan, tilt, zoom parameters that are herein defines as PTZ coordinates) to the captured images (image or pixel coordinates)”, discloses coordinates and position of the camera are transmitted to the server.)
Hanna does not explicitly teach a second server device, and determining, by the second server device, estimated candidate coordinates of coordinates, at which the target object is located, in 3D space based on 3D map information and the angle information received from the first server device.
Choi teaches a second server device(Fig. 6, 630 – shows a first server and 640 shows a second server.), and determining, by the second server device, estimated candidate coordinates of coordinates, at which the target object is located, in 3D space based on 3D map information and the angle information received from the first server device. (¶[0039], “the set of attributes of the at least one background object of the scene can include location information describing at least one location of the at least one background object represented by the 2D image data, and/or at least one measure of distortion of the at least one background object based on a 2D projection of the scene. 2D projection refers to the process of mapping points from 3D space onto respective points of a 2D plane, which can be used to transform the 3D scene into the 2D image data captured by the camera 120.”, ¶[0039] discloses estimating 3d coordinates of a target object using the 2d image acquired and ¶[0043] “In some embodiments, the output of the camera pose identification model is used to as an input to generate a 3D representation of the subject depicted in the 2D image data. In some embodiments, the engine 140 can analyze at least one movement of the subject by using the 3D representation (e.g., evaluating movement and changes in movement).”, discloses generating a 3d representation using the image acquired from the camera system.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Hanna with Choi in order to using 3d coordinates to determine the location of the object. One skilled in the art would have been motivated to modify Hanna in this manner in order to combine data channels to determine camera pose. (Choi, ¶[0002])
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Hanna et al. US PG-Pub(US 20100013917 A1) in view of Choi et al. US PG-Pub(US 20250037297 A1) in view of Ikeda et al. US PG-Pub(US 20250203194 A1).
Regarding Claim 5, while the combination of Hanna and Choi teach the electronic system of claim 1, they do not explicitly teach wherein the motor includes: an angle converter configured to rotate the camera in the pan rotation direction by the first angle and in the tilt rotation direction by the second angle; and a motor controller configured to determine the first angle and the second angle based on a structure of the angle converter and the angle control signal.
Ikeda teaches wherein the motor includes: an angle converter configured to rotate the camera in the pan rotation direction by the first angle and in the tilt rotation direction by the second angle; and a motor controller configured to determine the first angle and the second angle based on a structure of the angle converter and the angle control signal. (¶[0229]) “For example, the pan, tilt, and zoom setting (changing) processing is executed so that the angle of view including a face region, an upper body region, or an entire body region of a person set as the tracking target is obtained. As the pan, tilt, and zoom settings (changes), for example, the PTZ camera 50 controls a drive position (rotation angle with respect to the reference position) in the horizontal direction (pan direction) of the lens with which the PTZ camera 50 captures an image, controls a drive position (rotation angle with respect to the reference position) in the vertical direction (tilt direction) of the lens, controls a position (zoom magnification) at which the zoom lens of the PTZ camera 50 is moved in the optical axis direction, and controls a drive speed of the lens related to the pan, tilt, and zoom settings.”, ¶[0229] discloses controlling the PTZ camera based on the rotation angle of the reference position.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Hanna and Choi with Ikeda in order to control the pan and tilt direction of the camera system. One skilled in the art would have been motivated to modify [41] in this manner in order to improve the image quality of a clipped image. (Ikeda, ¶[0001])
Claims 6-7 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Hanna et al. US PG-Pub(US 20100013917 A1) in view of Choi et al. US PG-Pub(US 20250037297 A1) in view of Gupta et al. US Patent(US 11417069 B1).
Regarding Claim 6, the combination of Hanna and Choi teach the electronic system of claim 1, where Choi further teaches wherein the first server device provides the second server device with the angle information and a request signal for receiving the candidate coordinates(¶[0107], “Executable instructions 1022 can further be transmitted or received over a network via network interface device 1008. Executable instructions 1022 can include one or more of the engine 140, the engine 651, the training set generator 631, or the training engine 641.”, ¶[0107] discloses transmitting information between devices) , wherein the second server device includes a second processor including: a candidate region calculation module configured to determine a candidate region of the target object based on the angle information and the 3D map information in response to the request signal ([0046], “The camera pose identification model can be trained on a number of 2D images and/or 2D videos where the desired outputs have been independently measured. These include examples like a simple walking movement, where the subject begins at one fixed position and walks to another fixed position, and the locations of the two fixed positions and the camera 120 are known in advance. An example of simple walking movement is described below with reference to FIGS. 4-5.
[0047] Additionally, the locations and 3D shapes of any background objects observed within the scene may be known in advance.”, ¶[0046]-¶[0047] using machine learning to determine the position of the candidate object in the image.);
However, Hanna and Choi do not explicitly teach a candidate coordinate calculation module configured to determine the candidate coordinates based on the candidate region and the 3D map information, and wherein the candidate region indicates a location at which the second server device estimates that the target object is present in 3D space within the 3D map information
Gupta teaches a candidate coordinate calculation module configured to determine the candidate coordinates based on the candidate region and the 3D map information, and wherein the candidate region indicates a location at which the second server device estimates that the target object is present in 3D space within the 3D map information. (Col 3, Lines 45-53, “generating, using a pose estimation module, the at least one image, the camera location, the camera orientation, the camera distance to the object, and the bounding box of the object in each image: a pose of the object in the real world coordinates of the real world coordinates of the real 3D space; and outputting the pose of the object for the generating the 3D map which includes the object having the pose in the real world coordinates in the virtual 3D space.”, this section of the prior art discloses calculating the coordinates of the object in 3d and generating a 3d map to estimate the position of the object.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Hanna and Choi with Gupta in order to generate a 3d map with the 3d position of the object. One skilled in the art would have been motivated to modify [41] in this manner in order for real object and camera localization for interactive 3D mapping applications. (Gupta, Col 1, Lines 7-9)
Regarding Claim 7, the combination of Hanna, Choi and Gupta teach the electronic system of claim 6, where Hanna further teaches wherein the candidate coordinate calculation module determines candidate coordinates located at regular intervals within the candidate region, and wherein the candidate coordinates indicate estimated latitudes, longitudes, and altitudes of the coordinates at which the target object is located, respectively. ([0029] “The GUI set-up display 106 establishes a reference image (hereinafter referred to as a Zone Map) to establish a baseline of the area under surveillance. Specifically, the GUI set-up display 106 captures a series of images which may be segmented into a series of customized regions which are assigned various detection thresholds for detecting moving objects. Two-dimensional (X,Y) coordinates defining said regions form part of a look-up table of values that are mapped to PTZ coordinates,” ¶[0029] discloses setting series of regions in the image to determine the coordinates of the target object.)
Regarding Claim 11, it is substantially similar to claim 6 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Allowable Subject Matter
Claims 2-4, 8, 10 and 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 2 and 10, the primary reason for the allowance of the claims is the inclusion of the limitations, “wherein the camera device is configured to: provide the first server device with second image data corresponding to a second scene including the target object, and wherein the first server device includes a first processor including: an image processing module configured to recognize the target object in the second image data and to detect pixel coordinates of the target object; and a motor control module configured to calculate a pixel error corresponding to a result of comparing the pixel coordinates with center pixel coordinates of the second image data, and to generate the angle control signal based on the pixel error.”, in all the claims which is not found in the prior art references. It is noted that the examiner has not found any other prior art to anticipate or obviate the quoted claim limitations supra, when read in light/combination of the other claimed limitations within the cited claims. Also, it is noted that the quoted limitations, in combination with the other claim limitations of the cited claims, deem the claims patentable, not just the consideration of the quoted limitations by themselves.
Regarding Claim 3-4, these claims would be allowable by virtue of dependency on claim 2.
Regarding claims 8 and 12, the primary reason for the allowance of the claims is the inclusion of the limitations, “wherein the first server device is configured to: calculate altitude angles between the candidate coordinates, coordinates of the camera device, and altitude correction coordinates of the candidate coordinates, respectively; calculate angle errors between the altitude angles and the second angle, respectively; and determine one of the candidate coordinates corresponding to a smallest one of the angle errors as the coordinates at which the target object is located, and wherein the altitude correction coordinates indicate coordinates corrected such that altitudes of the candidate coordinates are identical to an altitude of the camera device.”, in all the claims which is not found in the prior art references. It is noted that the examiner has not found any other prior art to anticipate or obviate the quoted claim limitations supra, when read in light/combination of the other claimed limitations within the cited claims. Also, it is noted that the quoted limitations, in combination with the other claim limitations of the cited claims, deem the claims patentable, not just the consideration of the quoted limitations by themselves.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAN D HOANG whose telephone number is (571)272-4344. The examiner can normally be reached Monday-Friday 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JOHN M VILLECCO can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAN HOANG/Primary Examiner, Art Unit 2661
Prosecution Insights