DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “plurality of pieces of predefined action scenario information” in line 4 and “the plurality of pieces of action scenario information” in lines 8-9. It is unclear to the Examiner whether the limitation in line 4 is the same or different from the limitation in lines 8-9.
Claim 4 recites “the plurality of pieces of action scenario information” in lines 3-4 and claim 1 recites “plurality of pieces of predefined action scenario information” in line 4. It is unclear to the Examiner whether the limitation in lines 3-4 of claim 4 is the same or different from the limitation in line 4 of claim 1.
Claims 2, 3, 5-8, depend on at least claim 1. Therefore, the claims 2, 3, 5-8 are rejected for at least the same reason as claim 1.
Claims 9 and 12 recite similar limitations as claims 1 and 4, respectively. Therefore, claims 9 and 12 require similar corrections as claims 1 and 4, respectively.
Claim 16 recites “the device of claim 15” in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claims 10, 11, 13-15 depend on at least claim 9. Therefore, the claims 10, 11, 13-16 are rejected for at least the same reason as claim 9.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 5, 6, 9, 10, 13, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kallakuri et al. (U.S. Patent Application 20210409648) in view of Tu et al. (VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment) and further in view of Phipps et al. (U.S. Patent Application 20150195509).
In regards to claim 1, Kallakuri teaches a system [Fig. 1; e.g. system, 0073] for providing a multi-view video [e.g. a plurality of cameras sending sequences of images to the classification engine, 0026] artificial intelligence (AI)-based three-dimensional real space [Fig. 1; e.g. machine learning-based three-dimensional real space, 0007], the system comprising:
a memory [e.g. storage medium, 0023] configured to store measurement information of an indoor space [e.g. receiving an initial camera coverage plan including a three-dimensional map of a three-dimensional real space, 0007] and a plurality of pieces of predefined action scenario information [e.g. simulated subjects move through the space, 0011]; and
a processor configured to generate specification information about cameras [e.g. iteratively applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints, 0007] and a capacity of the indoor space using the measurement information of the indoor space [e.g. determining a set of camera coverage maps per camera including one of a second set of occupied voxels representing positions of simulated subjects on a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move through, 0011] and calculating the plurality of pieces of action scenario information before the three-dimensional real space built on the basis of the specification information is used [e.g. Calculating the poses of these cameras before installing is important to improve the accuracy and efficiency of the camera installation, 0224].
Kallakuri does not explicitly teach a studio platform, the system comprising:
acquire training data for estimating three-dimensional (3D) poses of users in a studio from simulation results.
However, Tu teaches the system comprising:
acquire training data for estimating three-dimensional (3D) poses of users in a three-dimensional real space from simulation results [e.g. we place a number of 3D poses (sampled from the motion capture datasets) at random locations in the space and project them to all views to get the respective 2D locations. Then we generate 2D heatmaps from the locations to train CPN, see section titled “Evaluation of CPN” in page 9].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Kallakuri’s system with the features of acquire training data for estimating three-dimensional (3D) poses of users in a studio from simulation results in the same conventional manner as taught by Tu because Tu provides a method which directly operates in the 3D space by gathering information from all camera views in order to avoid making incorrect decisions for each camera view [see section titled “Introduction” in page 2].
Kallakuri as modified by Tu does not explicitly teach a studio platform and a studio.
However, Phipps teaches a studio platform and a studio [Fig. 1; e.g. The studio includes a three dimensional motion capture system and two exemplar two dimensional studio cameras. The studio corresponds to the studio platform, whereas the physical studio space and its cameras correspond to the claimed studio, 0022].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified the combination of Kallakuri’s three-dimensional real space and the teachings of Tu with the features of a studio platform and a studio in the same conventional manner as taught by Phipps because studio platforms and studios are well known and commonly used in the art of motion capture systems [0003-0005].
In regards to claim 2, Kallakuri teaches the system of claim 1, wherein the processor acquires intrinsic [e.g. In internal calibration, the internal parameters of the cameras 114 are calibrated. Examples of internal camera parameters include focal length, principal point, skew, fisheye coefficients, etc., 0103] and extrinsic parameters [e.g. In external calibration, the external camera parameters are calibrated in order to generate mapping parameters for translating the 2D image data into 3D coordinates in real space, 0104] of each of cameras installed in the studio using a multi-view camera calibration technology [e.g. internal and external calibration of cameras, 0103-0104] on the basis of the specification information and acquires calibration [e.g. internal and external calibration, 0103-0104] and common coordinate systems for an indoor space of the three-dimensional real space [e.g. An arbitrary point in the real space, for example, the end of a shelf in one corner of the real space, is designated as a (0, 0, 0) point on the (x, y, z) coordinate system of the real space, 0109] using the intrinsic and extrinsic parameters of each of the cameras.
Kallakuri as modified by Tu does not explicitly teach the studio.
However, Phipps teaches the studio [Fig. 1; e.g. The studio includes a three dimensional motion capture system and two exemplar two dimensional studio cameras. The studio corresponds to the studio platform, whereas the physical studio space and its cameras correspond to the claimed studio, 0022].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified the combination of Kallakuri’s three-dimensional real space and the teachings of Tu with the features of the studio in the same conventional manner as taught by Phipps because studio platforms and studios are well known and commonly used in the art of motion capture systems [0003-0005].
In regards to claim 5, Kallakuri teaches the system of claim 1, wherein the specification information includes at least one of a minimum number of cameras [e.g. using a camera minimization criterion, the system can generate camera placement plans that reduce (or minimize) the number of cameras, 0235], disposition positions of the cameras [e.g. The final camera placement is defined as a set of 6D poses for the cameras with respect to the defined store origin. Each camera pose has the position (x, y, z) and the orientation (rx, ry, rz), 0217], and the capacity [e.g. determining a set of camera coverage maps per camera including one of a second set of occupied voxels representing positions of simulated subjects on a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move through, 0011].
In regards to claim 6, Kallakuri teaches the system of claim 1, wherein the specification information is matched to a plurality of pieces of predetermined indoor space measurement information [e.g. inventory display structure locations, entrances, exits, and designated unmonitored locations are mapped onto the 2D and 3D maps using the maps database, 0113-0114] and stored in the memory [e.g. storage medium storing instructions for determining an improved camera coverage plan including a number, a placement, and a pose of cameras, 0023, 0113-0114], and the studio is built on the basis of the specification information stored in the memory [e.g. providing the improved camera coverage plan to an installer to arrange cameras to track puts and takes of items by subjects in the three-dimensional real space, 0007].
In regards to claim 9, the claim recites similar limitations as claim 1 but in method form. Therefore, the same rationale as claim 1 is applied.
In regards to claim 10, the claim recites similar limitations as claim 2. Therefore, the same rationale as claim 2 is applied.
In regards to claim 13, the claim recites similar limitations as claim 5. Therefore, the same rationale as claim 5 is applied.
In regards to claim 14, the claim recites similar limitations as claim 6. Therefore, the same rationale as claim 6 is applied.
Claim(s) 3, 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kallakuri et al. (U.S. Patent Application 20210409648) in view of Tu et al. (VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment) and further in view of Phipps et al. (U.S. Patent Application 20150195509) as applied to claims 2, 10 above, and further in view of Liu et al. (U.S. Patent Application 20120148145).
In regards to claim 3, Kallakuri as modified by Tu and Phipps does not explicitly teach the system of claim 2, wherein the processor performs training for estimating the 3D poses of the users in the studio using the training data and the intrinsic and extrinsic parameters of each of the cameras.
However, Liu teaches the system [e.g. system, 0055] of claim 2, wherein the processor [e.g. vision processor, 0031] performs training [e.g. training, 0052] for estimating the 3D poses of the users [e.g. computing runtime object image’s 3D poses, 0055] in the three-dimensional real space [e.g. 3D space, 0080] using the training data [e.g. 3D training model positions, 0055] and the intrinsic and extrinsic parameters of each of the cameras [e.g. intrinsics and extrinsics (multi-camera calibration results), 0055].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified the combination of Kallakuri’s system and the teachings of Tu and Phipps with the features of wherein the processor performs training for estimating the 3D poses of the users in the three-dimensional real space using the training data and the intrinsic and extrinsic parameters of each of the cameras in the same conventional manner as taught by Liu because Liu provides method for computing the affine transformation among multi-telecentric camera system based on the multi-camera calibration information (including the camera intrinsic parameters and extrinsic parameters), and finding corresponding features among multiple non-perspective cameras in a significantly more accurate and efficient way [0079].
Kallakuri as modified by Tu and Liu does not explicitly teach the studio.
However, Phipps teaches the studio [Fig. 1; e.g. The studio includes a three dimensional motion capture system and two exemplar two dimensional studio cameras. The studio corresponds to the studio platform, whereas the physical studio space and its cameras correspond to the claimed studio, 0022].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified the combination of Kallakuri’s three-dimensional real space and the teachings of Tu and Liu with the features of the studio in the same conventional manner as taught by Phipps because studio platforms and studios are well known and commonly used in the art of motion capture systems [0003-0005].
In regards to claim 11, the claim recites similar limitations as claim 3. Therefore, the same rationale as claim 3 is applied.
Claim(s) 4, 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kallakuri et al. (U.S. Patent Application 20210409648) in view of Tu et al. (VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment) and further in view of Phipps et al. (U.S. Patent Application 20150195509) as applied to claims 2, 9 above, and further in view of Rowell et al. (U.S. Patent Application 20200342652).
In regards to claim 4, Kallakuri as modified by Tu and Phipps does not explicitly teach the system of claim 1, wherein the training data is multi-view video training data acquired from each of cameras installed in the studio regarding actions performed by at least one user on the basis of the plurality of pieces of action scenario information.
However, Rowell teaches the system [e.g. system, 0019] of claim 1, wherein the training data is multi-view video training data [e.g. Training datasets including multiple camera views may also be used to generate additional image data, 0168] acquired from each of cameras installed in the 3D scene [e.g. 3D scene, 0019] regarding actions [e.g. action scene (i.e., scenes including animations moving one or more objects), 0169] performed by at least one user [e.g. one or more objects such as humans, 0129, also see 0060] on the basis of the plurality of pieces of action scenario information [e.g. library of scene metadata, 0111, 0113].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified the combination of Kallakuri’s system and the teachings of Tu and Phipps with the features of wherein the training data is multi-view video training data acquired from each of cameras installed in the 3D scene regarding actions performed by at least one user on the basis of the plurality of pieces of action scenario information in the same conventional manner as taught by Rowell because Rowell provides a method for executing one or more routines to virtually perform scene creation, scene capture, and generation of additional image data channels allows the synthetic image generation system to produce vast quantities of image data at a fraction of the time and cost of conventional methods [0020].
Kallakuri as modified by Tu and Rowell does not explicitly teach the studio.
However, Phipps teaches the studio [Fig. 1; e.g. The studio includes a three dimensional motion capture system and two exemplar two dimensional studio cameras. The studio corresponds to the studio platform, whereas the physical studio space and its cameras correspond to the claimed studio, 0022].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified the combination of Kallakuri’s three-dimensional real space and the teachings of Tu and Rowell with the features of the studio in the same conventional manner as taught by Phipps because studio platforms and studios are well known and commonly used in the art of motion capture systems [0003-0005].
In regards to claim 12, the claim recites similar limitations as claim 4. Therefore, the same rationale as claim 4 is applied.
Allowable Subject Matter
Claims 7, 8, 15, 16 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
In regards to claim 7, the prior art of record fails to teach or suggest the system of claim 1, wherein the processor stores information on correct actions of experts suitable for a purpose of the studio in the memory in advance and performs classification and analysis on actions of each of the users on the basis of the information on the correct actions stored in the memory and 3D pose estimation results for each of the users in the studio.
In regards to claim 8, the claim depends on claim 7. Therefore, the claim is allowable for at least the same reason as claim 7 if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
In regards to claim 15, the claim recites similar limitations as claim 7. Therefore, the claim is allowable for at least the same reason as claim 7 if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
In regards to claim 16, the claim depends on claim 15. Therefore, the claim is allowable for at least the same reason as claim 15 if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW SHIN whose telephone number is (571)270-5764. The examiner can normally be reached Monday - Friday from 11:00AM to 7:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW SHIN/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights