DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicants Amendments filed on March 16, 2026, has been entered and made of record.
Currently pending Claim(s): 1-20
Independent Claim(s): 1, 9, 20
Amended Claim(s): 1, 3, 7-9, 12, 17-18, 20
Canceled Claim(s): 6, 16
Response to Arguments
This office action is responsive to the Applicant’s Arguments/Remarks Made in an Amendment
received on March 16, 2026.
In view of amendments filed on, the Applicant has amended independent Claim 1 to recite the limitations of Claim 6. Claims 9 and 20 have been similarly amended to include these limitations. Originally (in the Claim set dated January 17, 2024), Claim 1-2 and 6-11 were rejected over Chan (US Pub No 2020/0387718), in view of Zhang (Zhang, Qi; Chan, Antoni B, “3D Crowd Counting via Multi-View Fusion with 3D Gaussian Kernels”, 2020), and further in view of Jiang (US Pub No 2020/0242777).
The applicant explained (on pg. 12 paragraph 3, of Applicant’s Arguments/Remarks) that Zhang in particular fails to teach the limitation of “by the processing device, generating an aggregated volume model by projecting the plurality of 2D density maps according to a plurality of image capturing data”. The Applicant explained that Zhang teaches that “the source data used to generate the 3D density maps are the 2D camera-view features, not the 2D density maps”, and thus does not teach claimed limitation. The applicant further argued that the 2D density maps are used only to benefit the training of the feature extraction.
The Examiner respectfully disagrees with the Applicant’s assertion. The Examiner agrees that the 2D feature maps are projected to create a 3D fusion feature which is used to create the 3D prediction. However, Zhang also teaches the 2D density maps are generated from the 2D features maps (see pg. 2, Section 1, “2D single-view features are extracted and then decoded to the 2D density map”). Zhang further teaches that the 2D density maps may be projected (see pg. 12-13, Section 3.4, “The projection consistency between the projected 2D density maps and the 2D density map ground-truth is measured and used as part of the loss to further enhance the 3D counting performance”). These 2D density maps are then compared to 2D density maps created by projecting the 3D prediction. The initial 3D prediction is then corrected to create a refined 3D prediction , which is interpreted as the aggregated volume (see pg. 7, Section 5, “The projection consistency measure between the 3D prediction and 2D density map ground-truth is studied and then utilized in the loss function to refine the 3D prediction further”).
Thus, the final ‘aggregated volume’ can only be obtained by after the projection consistency measure. Thus, the Examiner has concluded that generating a refined 3D prediction through projecting 2D density maps meets the BRI of the limitation ‘generating an aggregated volume model by projecting the plurality of density maps’, since the final 3D prediction is obtained through the projection consistency measure, which comprises projected 2D density maps.
For all of the above reasons, the Examiner maintains the rejection of Claims 1, 9 and 20 by Chan, Zhang, and Jiang. If the applicant believes that this limitation distinguishes the current application over the prior art the Examiner suggests further amending the claim to more specifically state how the projection of the 2D density maps is linked to the generation of the claimed aggregated volume model.
Claims 1-2, 7-11, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chan et al. (US Pub No 2020/0387718), hereinafter Chan, in view of Zhang et al. (Zhang, Qi; Chan, Antoni B, “3D Crowd Counting via Multi-View Fusion with 3D Gaussian Kernels”, 2020), and further in view of Jiang et al. (US Pub No 2020/0242777), hereinafter Jiang.
As to Claim 1, Chan teaches an area information estimation method applicable to an area information estimation system (see paragraph [0001], “The present invention relates to a system and method for counting objects, and particularly, although not exclusively, to a system and method for counting number of people in an area”),
comprising a processing device (see Fig. 1, processing unit 102)
and a plurality of monitor devices (see Fig. 2, cameras 208)
and comprising: by the plurality of monitor devices, capturing a plurality of images of an area from different views (see paragraph [0026], “a plurality of cameras arranged to obtain a plurality of images representing the objects to be counted in a target area”);
generating a plurality of two-dimensional (2D) density maps of at least one target object in the area according to the plurality of image (see paragraph [0010], “In an embodiment of the first aspect, the step of generating the map for each of the plurality of images comprises the step of generating a plurality of view-level density maps representing a count of the objects in each of the scenes representing respective portions of the target area”).
Chan fails to teach by the processing device, generating an aggregated volume model by projecting the plurality of 2D density maps according to a plurality of image capturing data. Chan fails to further fails to teach generating a 3D density map according to the aggregated volume model, and then calculating a number of the at least one target object according to the 3D density map.. Instead, the 2D density maps are projected to the same 2D level of a 3D scene (see paragraph [0010]).
However, Zhang et al. teaches a method for obtaining a 3D density map which comprises obtaining multiple 2D density maps from 2D features (see pg. 2, Section 1, “2D single-view features are extracted and then decoded to the 2D density map”),
projecting the 2D features to obtain an initial aggregated model (see pg. 2, caption under Figure 2, “Single-view features are extracted and then projected to the 3D world on multiple height planes. The projected 3D features are concatenated and fused to output the 3D density map prediction”)
and then generating a refined aggregated volume model by projecting 2D density maps (see pg. 12-13, Section 3.4, “The projection consistency between the projected 2D density maps and the 2D density map ground-truth is measured and used as part of the loss to further enhance the 3D counting performance”), and see pg. 7, Section 5, “The projection consistency measure between the 3D prediction and 2D density map ground-truth is studied and then utilized in the loss function to refine the 3D prediction further”).
generating the 3D density map according to the aggregated volume model (see Zhang, Section 3, page 3, “3) 3D fusion and prediction: the projected multi-view 3D features are fused to predict the 3D density maps using 3D CNN layers”, and see Figure 2, page 2, where the 3D volume model created by projecting the features forms a 3D density map prediction).
Zhang further teaches calculating a number of the at least one target object according to the 3D density map (see Section 1, page 2, “we propose to use 3D projection and 3D feature fusion to perform the multi-view counting task”).
Zhang is combinable with Chan as both use image analysis to count objects. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Chan with the teachings of Zhang. The motivation for doing so would be to reduce errors in counting due to 2D-to-2D projecting. Zhang teaches in Section 1, page 2, “The disadvantage of the 2D-to-2D projection in (Zhang and Chan 2019) is that the features of the same person from different views may not line up correctly due to the approximation that all features come from the same height in the 3D world. Clearly this is not true for features extracted from the heads and feet of the people. To address this problem, in this paper, we propose to use 3D projection and 3D feature fusion to perform the multi-view counting task.”
Both Chan and Zhang fail to teach that the monitor device itself generates a plurality of 2D density maps of at least one target object in the area according to the plurality of images. Instead, images are sent to a computing device, which contains a processor which generates 2D density maps (see Chan, paragraph [0063]).
However, Jiang teaches a processor which can create 2D density maps (see paragraph [0004]), and can be integrated within a camera (see paragraph [0061] “For example, the system 600 may comprise, or be comprised in, an apparatus, such as a mobile phone, smart phone, camera (e.g., OzO, closed circuit television, webcam), drone, self-driving vehicle, car, unmanned aerial vehicle, autonomous vehicle, and/or Internet of Things (IoT sensor, such as a traffic sensor, industrial sensor, and/or the like) to enable counting of objects, in accordance with some example embodiments”). Jiang is combinable with Chan and Zhang since all three are from the analogous field of image analysis and crowd counting. Thus, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to combine the teachings of Jiang with Chang. The motivation for doing so would be to allow for the function of counting objects to be integrated into a wide variety of appliances. Thus, it would have been obvious to combine in the teachings of Chan with Jiang in order to obtain the invention as claimed in Claim 1.
As to Claim 2, Chan in view of Zhang and Jiang teaches wherein generating the plurality of 2D density maps of the at least one target object according to the plurality of images comprises:
by the plurality of monitor devices (see Jiang, paragraph [0061]),
transforming the plurality of images into the plurality of 2D density maps by using a 2D neural network model (see Chan ,paragraph [0088], “A fully-convolutional network (denoted as FCN-7) may be used on each camera view to extract image feature maps or estimate a corresponding view-level density map. ”).
As to Claim 7, Chan in view of Zhang and Jiang teaches calculating a position of at least one characteristic pixel point of the plurality of 2D density maps in the aggregated volume model according to the plurality of image capturing data, so as to form at least one voxel point of the aggregated volume model (see Zhang, Section 3.2, page 4, “Since each image pixel’s corresponding height in the 3D world is unknown, a height range H is used in the projection, where each pixel is projected to the 3D world multiple times onto different height planes. Then, the projected features from all height are concatenated along the z-dimension to form a 3D feature representation”, where a 3D feature representation implies multiple voxel positions are calculated).
As to Claim 8, Chan in view of Jiang fails to explicitly teach transforming the aggregated volume model into the 3D density map by using a 3D neural network model. However, Zhang teaches that a 3D neural network can be used to generate a 3D density map (see Section 3, page 3, “3D fusion and prediction: the projected multi-view 3D features are fused to predict the 3D density maps using 3D CNN layers”, where CNN stands for convolutional neural network).
As to Claim 9, Claim 9 recites a system, which performs the same method disclosed in Claim 1. Thus, the rejection and rationale are analogous to that made in Claim 1.
As to Claim 10, Chan teaches a plurality of monitor devices, each comprising a camera, configured to capture a corresponding one of the plurality of images (see paragraph [0026] “a plurality of cameras arranged to obtain a plurality of images representing the objects to be counted in a target area”).
Chan also teaches a processor (see Fig. 1, processing unit 102), configured to transform the corresponding one of the plurality of images into a corresponding one of the plurality of 2D density maps by using a 2D neural network model (see paragraph [0010], “In an embodiment of the first aspect, the step of generating the map for each of the plurality of images comprises the step of generating a plurality of view-level density maps representing a count of the objects in each of the scenes representing respective portions of the target area”).
Both Chan and Zhang fail to teach a monitor where the processor and camera are directly coupled together. However, Jiang teaches a processor which can create density maps (see paragraph [0004]), and can be coupled to a camera (see paragraph [0061] “For example, the system 600 may comprise, or be comprised in, an apparatus, such as a mobile phone, smart phone, camera (e.g., OzO, closed circuit television, webcam), drone, self-driving vehicle, car, unmanned aerial vehicle, autonomous vehicle, and/or Internet of Things (IoT sensor, such as a traffic sensor, industrial sensor, and/or the like) to enable counting of objects, in accordance with some example embodiments”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to combine the teachings of Jiang with Chan and Zhang. The motivation for doing so would be to allow for counting objects to be integrated into a wide variety of appliances. Thus, it would have been obvious to combine in the teachings of Chan with Jiang in order to obtain the invention as claimed in Claim 10.
As to Claim 11, Chan in view of Jiang teaches that the 2D neural network model is a convolutional neural network (see paragraph [0088], “A fully-convolutional network (denoted as FCN-7) may be used on each camera view to extract image feature maps or estimate a corresponding view-level density map”).
As to Claim 17, Claim 17 claims the same limitation claimed as Claim 7 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 7.
As to Claim 18, Claim 18 claims the same limitation claimed as Claim 8 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 8.
As to Claim 19, Chan in view of Jiang fails to explicitly teach the 3D neural network model is a convolutional neural network. However, Zhang teaches a 3D CNN can be used to transform an aggregated volume model into a 3D density map (see Section 3, page 3, “3) 3D fusion and prediction: the projected multi-view 3D features are fused to predict the 3D density maps using 3D CNN layers”, where CNN stands for convolutional neural network).
As to Claim 20, Claim 20 claims a non-transitory computer readable storage medium with a computer program (see Chan, paragraph [0134]) that when executed performs the same process disclosed in Claim 1. Therefore, the rejection and rationale are analogous to that made in Claim 1.
Claims 3-5 and 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Chan et al. (US Pub No 2020/0387718), hereinafter Chan, in view of Zhang et al. (Zhang, Qi; Chan, Antoni B, “3D Crowd Counting via Multi-View Fusion with 3D Gaussian Kernels”, 2020), and further in view of Jiang et al. (US Pub No 2020/0242777), hereinafter Jiang, and further in view of Yim et al. (US Pub No 2021/0374431), hereinafter Yim.
As to Claim 3, Chan in view of Zhang and Jiang fails to teach by the plurality of monitor providing a plurality of image capturing data corresponding to the plurality of images to the processing device.
However, Yim teaches device pose information can be obtained (see paragraph [0030], “Through an image calibration process of the camera, an intrinsic parameter and an extrinsic parameter may be obtained. The intrinsic parameter refers to internal information of the camera, such as structural errors of the camera caused by lens distortion or the like. The extrinsic parameter refers to information on how much the actually installed camera is moved from the origin of the world coordinate system and how much the camera is rotated”), and that this parameter information can be provided to processing units to perform distance calculation functions (see paragraph [0037], “According to another aspect, heads of two persons may be detected from the image captured by the camera, and a distance between the two persons may be calculated using a distance between positions of the heads on a plane corresponding to an average height using the camera parameters. In the video surveillance apparatus according to the aspect, the person-to-person distance calculation unit 100 may include a head-to-head distance calculation unit 150”).
Yim is combinable with Chan, Jiang, and Zhang since all four are from the analogous field of image analysis and surveillance. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yim with the teachings of Chan and Jiang. The motivation for doing so would be to better estimate the positions objects within images by using camera parameters. Yim teaches in paragraph [0024], “ When camera parameters are given, absolute coordinate information corresponding to an absolute distance may be obtained.” Thus, it would have been obvious to combine the teachings of Yim with the teachings of Chan, Jiang and Zhang in order to obtain the invention as claimed in Claim 3.
As to Claim 4, Chan in view of Zhang and Jiang fails to teach when at least one of the plurality of monitor devices is moved, the area information estimation method further comprises: by the at least one of the plurality of monitor devices, using a visual-based localization technology to calculate at least one device pose information so as to generate at least one of the plurality of image capturing data.
However, Yim teaches that device pose information can be obtained after a camera is moved (see paragraph [0030], “Through an image calibration process of the camera, an intrinsic parameter and an extrinsic parameter may be obtained. The intrinsic parameter refers to internal information of the camera, such as structural errors of the camera caused by lens distortion or the like. The extrinsic parameter refers to information on how much the actually installed camera is moved from the origin of the world coordinate system and how much the camera is rotated” and paragraph [0031], “ a camera image calibration technique based on a homography is proposed”, where homograph is used as a visual-based localization technology).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yim with the teachings of Chan, Zhang, and Jiang. The motivation for doing so would be to better estimate the positions objects within images by using camera parameters. Yim teaches in paragraph [0024], “ When camera parameters are given, absolute coordinate information corresponding to an absolute distance may be obtained.” Thus, it would have been obvious to combine the teachings of Yim with the teachings of Chan, Zhang, and Jiang in order to obtain the invention as claimed in Claim 4.
As to Claim 5, Chan in view of Zhang and Jiang fails to teach by the plurality of monitor devices, accessing a plurality of camera parameter information of a plurality of cameras of the plurality of monitor devices as the plurality of image capturing data. However, Yim teaches that camera parameter information can be obtained through the monitor device (see paragraph [0030], “Through an image calibration process of the camera, an intrinsic parameter and an extrinsic parameter may be obtained. The intrinsic parameter refers to internal information of the camera, such as structural errors of the camera caused by lens distortion or the like. The extrinsic parameter refers to information on how much the actually installed camera is moved from the origin of the world coordinate system and how much the camera is rotated”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yim with the teachings of Chan, Jiang, and Zhang. The motivation for doing so would be to better estimate the positions objects within images by using camera parameters. Yim teaches in paragraph [0024], “ When camera parameters are given, absolute coordinate information corresponding to an absolute distance may be obtained.” Thus, it would have been obvious to combine the teachings of Yim with the teachings of Chan, Zhang, and Jiang in order to obtain the invention as claimed in Claim 5.
As to Claim 12, Claim 12 claims the same limitation claimed as Claim 3 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 3.
As to Claim 13, Claim 13 claims the same limitation claimed as Claim 4 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 4.
As to Claim 14, Chan in view of Zhang teaches a plurality of monitor devices, wherein each monitor device comprises:
a camera, configured to capture a corresponding one of the plurality of images; (see paragraph [0026], “a plurality of cameras arranged to obtain a plurality of images representing the objects to be counted in a target area”).
Chan fails to teach that each monitor comprises a storage configured to store camera parameter information of the camera and a processor, coupled to the camera and the storage, and configured to access the camera parameter information as a corresponding one of the plurality of image capturing data.
However, Jiang teaches a computing system (see Fig.6, computing system 600) , which includes a processor and storage. Jiang further teaches that this computing system can be integrated into a camera (see paragraph [0061] “For example, the system 600 may comprise, or be comprised in, an apparatus, such as a mobile phone, smart phone, camera (e.g., OzO, closed circuit television, webcam), drone, self-driving vehicle, car, unmanned aerial vehicle, autonomous vehicle, and/or Internet of Things (IoT sensor, such as a traffic sensor, industrial sensor, and/or the like) to enable counting of objects, in accordance with some example embodiments”).
Jiang fails to teach that the processor and storage are configured to store camera parameter information and access camera parameter information.
However, Yim teaches that a processing unit (see Fig. 1, ground coordinate conversion unit 133) that uses stored camera parameters in order to calculate the positions of objects in images (see paragraph [0028], “The ground coordinate conversion unit 133 converts the pair of first position coordinates into a pair of second position coordinates of a ground coordinate system using the camera parameters”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yim with the teachings of Chan, Zhang and Jiang. The motivation for doing so would be to better estimate the positions objects within images by using camera parameters. Yim teaches in paragraph [0024], “ When camera parameters are given, absolute coordinate information corresponding to an absolute distance may be obtained.” Thus, it would have been obvious to combine the teachings of Yim with the teachings of Chan, Zhang, and Jiang in order to obtain the invention as claimed in Claim 14.
As to Claim 15, Chan in view of Zhang and Jiang teaches camera parameter information comprises camera intrinsic and camera extrinsic coefficients (see Chan, paragraph [0119], “The cameras' intrinsic and extrinsic parameters are estimated using a calibration algorithm”.
Chan in view of Jiang and Zhang fails to explicitly teach that distortion coefficients are included within the camera parameters. However, Yim teaches distortion parameters can be obtained (see paragraph [0031] “Through an image calibration process of the camera, an intrinsic parameter and an extrinsic parameter may be obtained. The intrinsic parameter refers to internal information of the camera, such as structural errors of the camera caused by lens distortion or the like. The extrinsic parameter refers to information on how much the actually installed camera is moved from the origin of the world coordinate system and how much the camera is rotated”
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yim with the teachings of Chan, Zhang, and Jiang. The motivation for doing so would be to correct for distortion to obtain more accurate measurements of distances between objects in images. Yim teaches in paragraph [0041], “ The direct camera-to-person distance calculation unit 170 detects heads of persons in a circle having a predetermined size based on a center of a screen in the image input from a wide angle camera 730 that is installed to face downward and converts the number of pixels between the detected heads of the persons into a distance to calculate the person-to-person distance…A radius of the circle may vary according to a distortion rate of a lens of the camera and software of the camera. The number of pixels between the central points of the head regions is proportional to the actual distance. Therefore, the distance may be calculated by multiplying the distance by the ratio value that varies according to the installation height of the camera.” Thus, it would have been obvious to combine the teachings of Yim with the teachings of Chan, Zhang, and Jiang in order to obtain the invention as claimed in Claim 15.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hu et al. (CN115249342) teaches obtaining multiple 2D density maps of a crowd, and then projecting the 2D density maps to obtain a 3D crowd map.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOUMYA THOMAS whose telephone number is (571)272-8639. The examiner can normally be reached M-F 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.T./Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights