Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “image acquisition part”, “vanishing point acquisition part”, “learning part”, “estimation part”, “calculation part”, and “output part” in claims 1, 3, 6, and 8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4-6, and 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2020/0126244 A1 of Lee (hereinafter referred to as Lee) in view of WO 2021/079456 of Gaku (hereinafter referred to as Gaku), and further in view of Lee, Jinwoo & Sung, Minhyuk & Lee, Hyunjoon & Kim, Junho. (2020). Neural Geometric Parser for Single Image Camera Calibration. 10.1007/978-3-030-58610-2_32. (hereinafter referred to as Jinwoo).
Lee discloses a learning part for performing a deep learning of deep neural networks using the image acquired by the image acquisition part and the coordinates of the plurality of true vanishing points acquired by the vanishing point acquisition part (see Lee par. [0051] “In operation 520, the training apparatus trains a first neural network (FCN 1) 603 to output first probability information about the vanishing point included in the training image 601, based on the training image 601 and the label corresponding to the manually generated GTD 602”); and an output part for outputting the deep neural networks learned in the learning part (see Lee par. [0084] “The deep learning training apparatus 1210 may generate an output of the neural network for trained data by using a training model in the GPU 1213”), wherein the learning part estimates coordinates of a plurality of vanishing points by inputting the image to the deep neural networks (see Lee par. [0032] “the detecting apparatus generates a probability map from the driving image by using a neural network.” and Lee par. [0036] “the detecting apparatus may determine the centroid of a candidate region for the vanishing point”) calculates a network error on the basis of the coordinates of the plurality of true vanishing points and the estimated coordinates of the plurality of vanishing points, and learns a parameter of the deep neural networks so as to minimize the calculated network error (see Lee par. [0067] “The training apparatus may train the FCN 1 810 such that loss 1 between the label 803 and the first candidate region of the vanishing point which is a detection result of the first regression module 820 is minimized.” And Lee Par. [0071] “The training apparatus may train the FCN 2 840 such that loss 2 between the probability map 807 corresponding to a modified label generated in the second regression module 830 and the second candidate region of the vanishing point detected in the third regression module 850 is minimized.”). Lee also discloses a non-transitory computer readable recording medium storing a learning program causing a computer to perform the above actions (see Lee par. [0087]).
Lee fails to disclose an image acquisition part for acquiring an image taken by a camera that causes a distortion and a vanishing point acquisition part for acquiring coordinates of a plurality of true vanishing points to calculate a tilt angle, a pan angle, and a roll angle of the camera and calculating error in the tilt angle, the pan angle, and the roll angle of the camera.
However, Gaku discloses an image acquisition part for acquiring an image taken by a camera that causes a distortion (see Gaku pg. 2 “The acquisition unit 11 is a ‘first coordinate pair’ extracted from ‘a plurality of person images’ included in one image in which the world coordinate space is photographed by a camera” and Gaku pg. 6 “Instead of the focal length, skew, optical center, or lens distortion may be used as an internal parameter to be estimated”, indicating that the camera lens contains distortion) and a vanishing point acquisition part for acquiring coordinates of a plurality of true vanishing points (see Gaku pg. 3 “The vanishing point calculation unit 12 calculates the "first vanishing point" in the horizontal direction based on the first coordinate pair and the second coordinate pair acquired by the acquisition unit 11, and the third coordinate acquired by the acquisition unit 11. The "second vanishing point" in the vertical direction is calculated based on the pair and the fourth coordinate pair.”).
It would have been obvious for a person having ordinary skill in the art to combine the neural network of Lee with the vanishing point acquisition of Gaku because it is predictable that doing so would improve the quality of the training data by enabling the system to obtain true vanishing points.
Furthermore, Jinwoo discloses calculating a tilt angle, a pan angle, and a roll angle of the camera (see Jinwoo pg. 4 “our network estimates up to four camera intrinsic and extrinsic parameters; the focal length f and three camera rotation angles ψ, θ, φ.”) and calculating error in the tilt angle, the pan angle, and the roll angle of the camera (see Jinwoo pg. 12 “Note that, for DeepHorizon [32]*, we use GT FoV to calculate the camera up vector (angle, pitch, and roll errors) from the predicted horizon line.”).
It would have been obvious for a person having ordinary skill in the art to combine the neural network of Lee and the vanishing point acquisition of Gaku with the tilt, pan, and roll angle calculator and error calculator of Jinwoo because it is predictable that doing so would allow the system to detect more camera parameters to assist in determining the calibration for the distortion in the image, as well as create an error metric to define and therefore increase accuracy.
Claims 4 and 5 are rejected according to the same analysis as claim 1 above.
Regarding claim 6, Lee discloses a camera parameter calculation device comprising: an estimation part for estimating coordinates of a plurality of vanishing points by inputting the image acquired by the image acquisition part to deep neural networks learned by a deep learning (see Lee par. [0032] “the detecting apparatus generates a probability map from the driving image by using a neural network.” And par. [0036] “the detecting apparatus may determine the centroid of a candidate region for the vanishing point”), a calculation part for calculating the tilt angle on the basis of the coordinates of the plurality of vanishing points estimated by the estimation part (see Lee par. [0041] “In operation 320, the detecting apparatus estimates a pitch variation based on the detected vanishing point”), and an output part for outputting a camera parameter including the tilt angle calculated by the calculation part (see Lee par. [0037] “the detecting apparatus may determine a pitch variation based on the vanishing point, and update (or correct) a transformation matrix based on the pitch variation” and Lee par. [0044] “the detecting apparatus converts the domain of the driving image into the domain of the top-view image based on the updated transformation matrix.”), wherein in the learning of the deep neural networks, a learning-use image is acquired, coordinates of a plurality of vanishing points used for taking the learning-use image are estimated by inputting the learning-use image to the deep neural networks (see Lee par. [0032] “the detecting apparatus generates a probability map from the driving image by using a neural network.” And par. [0036] “the detecting apparatus may determine the centroid of a candidate region for the vanishing point”), and a network error of the coordinates of the plurality of true vanishing points and the estimated coordinates of the plurality of vanishing points, and a parameter of the deep neural networks is learned so as to minimize the calculated network error (see Lee par. [0067] “The training apparatus may train the FCN 1 810 such that loss 1 between the label 803 and the first candidate region of the vanishing point which is a detection result of the first regression module 820 is minimized.” And Lee Par. [0071] “The training apparatus may train the FCN 2 840 such that loss 2 between the probability map 807 corresponding to a modified label generated in the second regression module 830 and the second candidate region of the vanishing point detected in the third regression module 850 is minimized.”). Lee also discloses a non-transitory computer readable recording medium storing a camera parameter calculation program causing a computer to perform the above actions (see Lee par. [0087]).
Lee fails to disclose an image acquisition part for acquiring an image taken by a camera that causes a distortion and coordinates of a plurality of true vanishing points to calculate a tilt angle, a pan angle, and a roll angle of a camera used for taking the learning-use image are acquired and calculating the error in the tilt angle, the pan angle, and the roll angle of the camera.
However, Gaku discloses an image acquisition part for acquiring an image taken by a camera that causes a distortion (see Gaku pg. 2 “The acquisition unit 11 is a "first coordinate pair" extracted from "a plurality of person images" included in one image in which the world coordinate space is photographed by a camera” and Gaku pg. 6 “Instead of the focal length, skew, optical center, or lens distortion may be used as an internal parameter to be estimated”, indicating that the camera lens contains distortion) and coordinates of a plurality of true vanishing used for taking the learning-use image are acquired (see Gaku pg. 3 “The vanishing point calculation unit 12 calculates the "first vanishing point" in the horizontal direction based on the first coordinate pair and the second coordinate pair acquired by the acquisition unit 11, and the third coordinate acquired by the acquisition unit 11. The "second vanishing point" in the vertical direction is calculated based on the pair and the fourth coordinate pair.”).
It would have been obvious for a person having ordinary skill in the art to combine the neural network of Lee with the vanishing point acquisition of Gaku because it is predictable that doing so would improve the quality of the training data by enabling the system to obtain true vanishing points.
Furthermore, Jinwoo discloses calculating a tilt angle, a pan angle, and a roll angle of the camera (see Jinwoo pg. 4 “our network estimates up to four camera intrinsic and extrinsic parameters; the focal length f and three camera rotation angles ψ, θ, φ.”), calculating error in the tilt angle, the pan angle, and the roll angle of the camera (see Jinwoo pg. 12 “Note that, for DeepHorizon [32]*, we use GT FoV to calculate the camera up vector (angle, pitch, and roll errors) from the predicted horizon line.”), and outputting the tilt angle, the pan angle, and the roll angle of the camera (see Jinwoo pg. 11 Table 1, which indicates output of Jinwoo’s system includes the camera rotation parameters).
It would have been obvious for a person having ordinary skill in the art to combine the neural network of Lee and the vanishing point acquisition of Gaku with the tilt, pan, and roll angle calculator and error calculator of Jinwoo because it is predictable that doing so would allow the system to detect more camera parameters to assist in determining the calibration for the distortion in the image, as well as create an error metric to define and therefore increase accuracy.
Claims 9 and 10 are rejected according to the same analysis as claim 6 above.
Allowable Subject Matter
Claims 2, 3, 7, and 8 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIO A. RODIN whose telephone number is (571)272-8003. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at 571-272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARIO ANTHONY RODIN/Examiner, Art Unit 2675
/ANDREW M MOYER/Supervisory Patent Examiner, Art Unit 2675
Prosecution Insights