DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/27/2025 in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-2, 5, 7-11, 14-16, 18-21 of U.S. Patent No. US 12382566 B2 in view of Matthew et al.( (Pub. No, US US 20160150143 A1), hereafter Matthew.
Below is the table of comparison between claims in cases involved in this double patenting rejection. Differences are underlined.
Subject Application Claim Text
Application # 19/253,248 (hereafter ‘248)
Conflicting Patent Claim Text
US Patent # 12,382,566 (hereafter ‘566)
1. Method for controlling a light source, the method using (a) at least one pose estimate of a camera configured to capture one or more images of a scene of interest which comprises at least one landmark, as said light source is operated to emit light which illuminates said scene of interest, (b) a landmark map comprising at least 3D location information of a plurality of landmarks comprising the at least one landmark in the scene of interest, (c) an illumination model describing a relationship between an emission illumination power and a reflection illumination power, wherein said emission illumination power is the power of light emitted by the light source to illuminate said scene of interest, and said reflection illumination power is the illumination power of light reflected by one or more landmarks in said scene of interest and received by the camera, wherein the method comprises the following steps:(1) determining, for at least one of the plurality of landmarks, at least one emission illumination power of light to be emitted by the light source, and an illumination time during which the light source should be operated to emit light which has an emission illumination power which is equal to the at least one emission illumination power, using (i) the at least one pose estimate of the camera, (ii) the 3D location information of the at least one of the plurality of landmarks, (iii) the illumination model; and (2) operating the light source to emit light which has an emission illumination power which is equal to the at least one emission illumination power, for a time period which is equal to the determined illumination time course; and (3) updating the illumination model parameters, wherein the illumination model parameters which are updated comprise reflectivity parameters of at least one landmark.
1. Method for controlling a light source , the method using (a) at least one pose estimate of a camera configured to capture one or more images of a scene of interest which comprises at least one landmark, as said light source is operated to emit light which illuminates said scene of interest, (b) a landmark map comprising at least 3D location information of a plurality of landmarks comprising the at least one landmark in the scene of interest, (c) an illumination model describing a relationship between an emission illumination power and a reflection illumination power, wherein said emission illumination power is the power of light emitted by the light source to illuminate said scene of interest, and said reflection illumination power is the illumination power of light reflected by one or more landmarks in said scene of interest and received by the camera, and (d) a predefined threshold reflection illumination power, wherein the method comprises the following steps:(la) determining, for at least one of the plurality of landmarks, at least one optimized emission illumination power of light to be emitted by the light source, wherein the optimized emission illumination power is a solution to an optimization problem for energy of the emitted light delivered in the predefined illumination time course, and an illumination time course during which the light source should be operated to emit light which has an emission illumination power which is equal to the at least one optimized emission illumination power, using (i) the at least one pose estimate of the camera, (ii) the 3D location information of the at least one of the plurality of landmarks, (iii) the illumination model, and (iv) the predefined threshold reflection illumination power; and(2b) operating the light source to emit light which has an emission illumination power which is equal to the at least one optimized emission illumination power, for a time period which is equal to the determined illumination time course.
2. The method according to claim l wherein the step (3) of updating the illumination model parameters comprises, updating the illumination model parameters based on a deviation between the predicted received illumination power and measured received illumination power.
17. Method according to claim 1, comprising updating the illumination model parameters based on a deviation between the predicted received illumination power and measured received illumination power.
3. Method according to claim 1, wherein the at least one emission illumination power is a solution to an optimization problem for energy of the emitted light delivered in the predefined illumination time course, so that said at least one emission illumination power is an optimized emission illumination power.
1. “at least one of the plurality of landmarks, at least one optimized emission illumination power of light to be emitted by the light source, wherein the optimized emission illumination power is a solution to an optimization problem for energy of the emitted light delivered in the predefined illumination time course”.
4. Method according to claim 1, wherein the at least one emission illumination power is a solution to an optimization problem for the uncertainty of a posterior first pose estimate, so that said at least one emission illumination power is an optimized emission illumination power.
7. “the first optimized emission illumination power is determined together with a posterior first pose estimate uncertainty of the camera, which posterior first pose estimate uncertainty is determined, together with the first optimized emission illumination power”.
5. Method according to claim 1, wherein the illumination model is configured to model a non-isotropically emitting light source.
15. Method according to claim 1, wherein the illumination model is configured to model a non-isotropically emitting light source, wherein the light source is embodied as a directional light source.
6. Method according to claim 1, wherein the at least one pose estimate comprises a first pose estimate, and wherein the determining of the at least one emission illumination power comprises determining a first emission illumination power by: (a) determining distances between the first pose estimate and the 3D location of the plurality of landmarks, (b) sorting the distances in an ascending order or descending order, (c) choosing an M-th distance from the sorted distances, and (d) using the M-th distance for determining the first emission illumination power using at least the illumination model.
2. Method according to claim 1, wherein the at least one pose estimate comprises a first pose estimate, and wherein the determining of the at least one optimized emission illumination power provides a first optimized emission illumination power, wherein the determining of the first optimized emission illumination power comprises as steps: (a) determining distances between the first pose estimate and the 3D location of the plurality of landmarks, (b) sorting the distances in an ascending order or descending order, (c) choosing an M-th distance from the sorted distances, and (d) using the M-th distance for determining the first optimized emission illumination power using at least the illumination model.
7. Method according to claim 1, wherein the at least one emission illumination power is determined using a constrained optimization algorithm with a predefined illumination time course, wherein the constrained optimization algorithm is configured to extremize a cost function while fulfilling constraints.
3. Method according to claim 1, wherein the at least one optimized emission illumination power is determined using a constrained optimization algorithm with a predefined illumination time course, wherein the constrained optimization algorithm is configured to extremize a cost function while fulfilling constraints.
8. Method according to claim 7, wherein the constrained optimization algorithm is configured to minimize or maximize the cost function, by varying at least the at least one emission illumination power.
4. Method according to claim 3, wherein the constrained optimization algorithm is configured to minimize or maximize the cost function, by varying at least the at least one emission illumination power.
9. Method according to claim 1, wherein the illumination model comprises illumination model parameters, wherein at least one of the illumination model parameters is a stochastic parameter, wherein determining the at least one emission illumination power involves stochastically propagating illumination model parameter uncertainty through the illumination model.
5. Method according to claim 1, wherein the illumination model comprises illumination model parameters, wherein at least one of the illumination model parameters is a stochastic parameter, wherein determining the at least one optimized emission illumination power involves stochastically propagating illumination model parameter uncertainty through the illumination model.
10. Method according to claim 1, wherein the at least one pose estimate comprises a first pose estimate and wherein the first pose estimate comprises a first pose estimate uncertainty, wherein a first emission illumination power is determined together with a posterior first pose estimate uncertainty of the camera, which posterior first pose estimate uncertainty is determined, together with the first emission illumination power, using at least (i) the first pose estimate and the first pose estimate uncertainty, (ii) the illumination model, and (iii) a localization model for determining a pose of the camera using positions of landmarks in an image acquired by the camera at the pose, wherein the first emission illumination power is determined in such a way that the posterior first pose estimate uncertainty is below a predefined posterior uncertainty threshold.
7. Method according to claim 1, wherein the at least one pose estimate comprises a first pose estimate and wherein the first pose estimate comprises a first pose estimate uncertainty, wherein the first optimized emission illumination power is determined together with a posterior first pose estimate uncertainty of the camera, which posterior first pose estimate uncertainty is determined, together with the first optimized emission illumination power, using at least (i) the first pose estimate and the first pose estimate uncertainty, (ii) the illumination model, and (iii) a localization model for determining a pose of the camera using positions of landmarks in an image acquired by the camera at the pose, wherein the first optimized emission illumination power is determined in such a way that the posterior first pose estimate uncertainty is below a predefined posterior uncertainty threshold.
11. Method according to claim 10, wherein the first emission illumination power is determined together with a second pose estimate uncertainty of the camera, which second pose estimate uncertainty is determined, together with the first emission illumination power, by additionally using a movement model of the camera, wherein the first emission illumination power is determined in such a way that the second pose estimate uncertainty, obtained by forward-projecting the posterior first pose estimate uncertainty using the movement model to a time tt2, tt2 > tt1, at which the camera is configured to capture a subsequent image, is below a predefined uncertainty threshold.
8. Method according to claim 7, wherein the first optimized emission illumination power is determined together with a second pose estimate uncertainty of the camera, which second pose estimate uncertainty is determined, together with the first optimized emission illumination power, by additionally using a movement model of the camera, wherein the first optimized emission illumination power is determined in such a way that the second pose estimate uncertainty, obtained by forward-projecting the posterior first pose estimate uncertainty using the movement model to a time t.2, t.2>t.1, at which the camera is configured to capture a subsequent image, is below a predefined uncertainty threshold
12. Method according to claim 1, wherein the first emission illumination power is determined by solving the following constrained optimization problem [FORMULA] wherein EETTTT is the energy of the emitted light delivered in the predefined illumination time course, ||-|| is a matrix norm, Xpp(EETTTT) is a covariance matrix relating to the posterior first pose estimate uncertainty, which covariance matrix depends on the energy of the emitted light, wherein the posterior first pose estimate is determined based on landmarks which are detectable in the image in case the emitted light has energy EETTTT, and wherein 00pp,mm is the predefined posterior uncertainty threshold, so that the said first emission illumination power is a first optimized emission illumination power.
9. Method according to claim 7, wherein the first optimized emission illumination power is determined by solving the following constrained optimization problem [FORMULA, wherein ETX is the energy of the emitted light delivered in the predefined illumination time course, ∥⋅∥ is a matrix norm, Σ.sub.p(ETX) is a covariance matrix relating to the posterior first pose estimate uncertainty, which covariance matrix depends on the energy of the emitted light, wherein the posterior first pose estimate is determined based on landmarks which are detectable in the image in case the emitted light has energy ETX, and wherein σ.p,max is the predefined posterior uncertainty threshold.
13. Method according to claim 12, wherein the constrained optimization algorithm is relaxed to an unconstrained optimization algorithm, (FORMULA) wherein Vt is a scaling parameter tending to infinity, so that said emission illumination power is an optimized emission illumination power.
10. Method according to claim 7, wherein the constrained optimization algorithm is relaxed to an unconstrained optimization algorithm (FORMULA)., wherein λ is a scaling parameter.
14. Method according to claim 7, wherein a first emission illumination power is determined by solving the following constrained optimization problem, [FORMULA] wherein EETTTT is the energy of the emitted light delivered in the predefined illumination time course, IIII is an index over at least a subset of the plurality of landmarks, wherein EERRTT, is a reflection illumination energy corresponding to the IIII-th landmark, and EERRTT, is a reflection illumination energy at which the camera saturates, so as to provide a first optimized emission illumination power.
11. Method according to claim 3, wherein the first optimized emission illumination power is determined by solving the following constrained optimization problem [FORMULA] wherein ETX is the energy of the emitted light delivered in the predefined illumination time course, Ix is an index over at least a subset of the plurality of landmarks, wherein ERX Ix is a reflection illumination energy corresponding to the Ix-th landmark, and ERX max is a reflection illumination energy at which the camera saturates.
15. Method according to claim 1, wherein the illumination model is embodied as follows: [FORMULA] wherein PPRRTT is the reflection illumination power, PPTTTT is the emission illumination power, ff is the focal length of the camera, NN is the f-number of the camera, RRAA is a landmark reflectivity of a reflector landmark, wherein at least one landmark of the plurality of landmarks is embodied as said reflector landmark, AARRis a projected surface area of said reflector landmark as viewed from the light source, and dd is the distance between the light source and said reflector landmark.
14. Method according to claim 13, wherein the illumination model is embodied as follows [FORMULA], wherein PRX is the reflection illumination power, PTX is the emission illumination power, f is the focal length of the camera, N is the f-number of the camera, RA is a landmark reflectivity of a reflector landmark, wherein at least one landmark of the plurality of landmarks is embodied as said reflector landmark, AR is a projected surface area of said reflector landmark as viewed from the light source, and d is the distance between the light source and said reflector landmark.
16. Method according to claim 1, wherein the determining of the at least one emission illumination power comprises comparing a predefined threshold reflection illumination power to a predicted received illumination power using the illumination model, wherein the at least one emission illumination power is set in such a way that a corresponding at least one predicted received illumination power is equal to or greater than the predefined threshold reflection illumination power.
16. Method according to claim 1, wherein the determining of the at least one optimized emission illumination power comprises comparing the predefined threshold reflection illumination power to a predicted received illumination power using the illumination model, wherein the at least one optimized emission illumination power is set in such a way that the corresponding at least one predicted received illumination power is equal to or greater than the predefined threshold reflection illumination power.
17. Method according to claim 1, wherein the at least one pose estimate comprises a plurality of pose estimates and wherein the at least one emission illumination power comprises a plurality of emission illumination powers, the plurality of pose estimates and emission illumination powers relating to a planned and/or predicted movement of the camera, and adapting the planned and/or predicted movement based on an output of the constrained optimization algorithm.
18. Method according to claim 1, wherein the at least one pose estimate comprises a plurality of pose estimates and wherein the at least one optimized emission illumination power comprises a plurality of optimized emission illumination powers, the plurality of pose estimates and optimized emission illumination powers relating to a planned and/or predicted movement of the camera, and adapting the planned and/or predicted movement based on an output of the constrained optimization algorithm.
18. Method according to claim 17, wherein the at least one emission illumination power is determined by solving the following constrained optimization problem:
[FORMULA],
wherein JJ is a natural number denoting that emission illumination power is to be minimized for JJ subsequent images and wherein emission illumination energies EETTTT, EETTTT, being the energy of the i-th emission of light, for the JJ subsequent images are arranged in a vector, and wherein the cost function is embodied as sum over the JJ emission illumination energies, and wherein gg is a constraint function embodied as vector-valued function or as scalar-valued function, and taking the vector comprising the JJ emission illumination energies as input, so that said emission illumination power is an optimized emission illumination power.
19. Method according to claim 18, wherein the at least one optimized emission illumination power is determined by solving the following constrained optimization problem,
[FORMULA]
wherein J is a natural number denoting that emission illumination power is to be minimized for J subsequent images and wherein emission illumination energies ETX,i, ETX,i being the energy of the i-th emission of light, for the J subsequent images are arranged in a vector, and wherein the cost function is embodied as sum over the J emission illumination energies, and wherein g is a constraint function embodied as vector-valued function or as scalar-valued function, and taking the vector comprising the J emission illumination energies as input.
19. A non-transitory storage medium containing computer program product comprising instructions which when executed by a computer, cause the computer to carry out a method according to claim 1.
20. A non-transitory storage medium containing computer program product comprising instructions which when executed by a computer, cause the computer to carry out a method according to claim 1.
20. Assembly, comprising (a) a light source, (b) a camera, (c) a plurality of landmarks, and (d) a controller which is configured to carry out a method according to claim 1.
21. Assembly, comprising (a) a light source, (b) a camera, (c) a plurality of landmarks, and (d) a controller which is configured to carry out a method according to claim 1.
Regarding claim 1, patent ‘566 (claim 1) recites all the limitations except the following feature as discussed below.
Patent ‘566 does not teach (3) updating the illumination model parameters, wherein the illumination model parameters which are updated comprise reflectivity parameters of at least one landmark.
Mathews teaches (3) updating the illumination model parameters, wherein the illumination model parameters which are updated comprise reflectivity parameters of at least one landmark (paragraph [0064], “a two-stage inverse lighting optimization technique is subsequently applied to each image 710 at operations 602-605 to recover lighting parameters that best reproduce the 3D reconstructed landmark's appearance (e.g. albedos, occlusions, and reflectance)”).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify Patent ‘566 in view of Matthews to incorporate model parameters which are updated comprise reflectivity parameters hence reflectance as in various implementations of this optimization technique, a rendering equation may be used to predict what the appearance of the object should be in an image given the object's geometry, albedos, and reflectance (Matthews, paragraph [0054]).
Regarding claim 11, all limitations of the subject application '248 are included in claim 8 of the stated patent ‘566. Formulas are same despite naming parameters differently.
Regarding claim 12, all limitations of the subject application '248 are included in claim 9 of the stated patent ‘566. Formulas are same despite naming parameters differently.
Regarding claim 13, all limitations of the subject application '248 are included in claim 10 of the stated patent ‘566. Formulas are same despite naming parameters differently.
Regarding claim 14, all limitations of the subject application '248 are included in claim 11 of the stated patent ‘566. Formulas are same despite naming parameters differently.
Regarding claim 15, all limitations of the subject application '248 are included in claim 14 of the stated patent ‘566. Formulas are same despite naming parameters differently.
Regarding claim 18, all limitations of the subject application '248 are included in claim 19 of the stated patent ‘566. Formulas are same despite naming parameters differently.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED M KAISER whose telephone number is (571)272-9612. The examiner can normally be reached M-F 9 a.m.-6 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Riyami can be reached at 571-270-3119. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED M KAISER/Examiner, Art Unit 2831 /ABDULLAH A RIYAMI/Supervisory Patent Examiner, Art Unit 2831
Prosecution Insights