DETAILED ACTION
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11, 13-17, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bolvin et al. (US 2015/0084951) In view of Someya (US 11218662).
[Claim 1] Bolvin teaches An information processing apparatus comprising:
circuitry configured to
obtain at least one first image (i.e. white circles on black background) of a display, the at least one first image being acquired by a first camera (i.e. witness camera or film camera, both can read the patterns) (p. 0239-0240),
obtain at least one second image (i.e. black circles on white background) of at least one marker (i.e. simple pattern detector) in an imaging space around the display, the at least one second image being acquired by a second camera (i.e. witness camera or film camera, both can read the patterns) (p. 0239-0240),
estimate first position information (i.e. using lineup algorithm) of the first camera in relation to the display (i.e. 3D offset) based on the at least one first image (i.e. pattern) (p. 0239-0240),
obtain offset information between the first camera and the second camera (i.e. 3D offset (p. 0239-0240), and
estimate second position information (i.e. positional information for witness camera or film camera from patterns) of the second camera and third position information of the at least one marker (i.e. natural markers) in relation to the display based on the first position information (i.e. utilizing lineup algorithm), the at least one second image (i.e. 2nd pattern), and the offset information (i.e. 3D offset) (p. 0189, 0200, 0239-0240). natural markers are automatically learned and computed in 3D by the system as the user scans the environment walking around. As soon as the user has started to do the survey, he has to also capture the registration object (see section 2.1.3.2) in order to automatically estimate the scale, origin and orientation of the world;
wherein a positional relation between the first camera and the second camera is fixed (i.e. witness and film cameras are attached to each other) (p. 0179, 0240).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 2] Bolvin teaches the information processing apparatus according to claim 1, wherein the circuitry is further configured to control output of a display image by the display according to a position of the first camera (i.e. projecting sensor depths of the film camera) (p. 0017).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 3] Bolvin teaches the information processing apparatus according to claim 2,
wherein the circuitry is further configured to control the output of the display image by the display during virtual production (i.e. virtual modeling) in which the display image output by the display is included in images acquired by the first camera (constructing depth parameters) (p. 0297).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 4] Bolvin teaches The information processing apparatus according to claim 1, wherein the at least one first image of the display includes a display marker (i.e. natural markers) (p. 0189).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 5] Bolvin teaches The information processing apparatus according to claim 4, wherein the display marker includes an augmented reality marker (i.e. augmented reality framework for natural markers) (p. 0192-0193).
[Claim 6] Bolvin teaches The information processing apparatus according to claim 4, wherein the display marker is displayed at known coordinates in a coordinate system of the display (i.e. calibration chart uses known coordinates) (p. 140, 0240-0244).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 7] Bolvin teaches The information processing apparatus according to claim 6, wherein the circuitry is configured to estimate the first position information of the first camera in relation to the display
using the known coordinates of the display marker (i.e. calibration chart uses known coordinates with natural markers) (p. 140, 0240-0244).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 8] Bolvin teaches The information processing apparatus according to claim 6, wherein the circuitry is further configured to control output of a display image by the display
according to a position and orientation (i.e. positional information) of the first camera in the coordinate system of the display (i.e. witness camera or film camera, both can read the patterns) (p. 0239-0240).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 9] Bolvin teaches The information processing apparatus according to claim 1, wherein the circuitry is configured to obtain a plurality of first images of the display (i.e. white or black circles) (p. 0240), and
wherein the circuitry is configured to obtain a plurality of second images of the at least one marker (i.e. natural markers) corresponding to the plurality of first images (i.e. pattern) (p. 0189, 0200, 0239-0240). natural markers are automatically learned and computed in 3D by the system as the user scans the environment walking around. As soon as the user has started to do the survey, he has to also capture the registration object (see section 2.1.3.2) in order to automatically estimate the scale, origin and orientation of the world.
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 10] Bolvin teaches The information processing apparatus according to claim 9, wherein the plurality of first images of the display are acquired by the first camera (i.e. film camera) from a position corresponding to the first position information (i.e. positional information) and the plurality of second images are acquired by the second camera (i.e. witness camera) from a position corresponding to the second position information (i.e. positional information whilst cameras attached to each other) (p. 0239-0240).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 11] Bolvin teaches The information processing apparatus according to claim 9, wherein the plurality of first images are acquired by the first camera (i.e. film camera) from a plurality of positions and the plurality of second images are acquired by the second camera (i.e. witness camera) from a plurality of positions corresponding to the plurality of positions of the first camera (i.e. using lineup algorithm to track objects or natural markers) (p. 0194, 0239-0240).
[Claim 13] Bolvin teaches The information processing apparatus according to claim 1, wherein the circuitry is configured to estimate the second position information of the second camera in relation to the display based on the first position information and the offset information (i.e. using lineup algorithm to track objects or natural markers and determine the 3D offset) (p. 0194, 0239-0240).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 14] Bolvin teaches The information processing apparatus according to claim 13, wherein the circuitry is configured to estimate the third position information (i.e. all position information utilized by the lineup algorithm) of the at least one marker in relation to the display based on the at least one second image and the second position information (i.e. using lineup algorithm to track objects or natural markers and determine the 3D offset) (p. 0194, 0239-0240).
Bolvin is not entirely clear in teaching the specific feature of:
“images of a display screen”.
Someya teaches the specific feature of:
“images of a display screen” (i.e. estimates tilt of the camera based on the projection surface) (col. 4, lines 25-33, col.17, lines 16-25).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided a display screen surface as taught by Someya to the system of Bolvin to provide a projection surface for camera calibration (col. 4, lines 25-33).
[Claim 15] Bolvin teaches The information processing apparatus according to claim 14, wherein the at least one marker includes a plurality of markers (i.e. natural markers) provided around the second camera in the imaging space, and wherein the circuitry is further configured to generate a map of the plurality of markers based on the third position information (i.e. creating a map of 3D point cloud markers) (p. 0224-0228).
[Claim 16] Bolvin teaches The information processing apparatus according to claim 1, wherein the obtained offset information (i.e. 3D offset) is obtained based on predetermined offset information (i.e. utilizing lineup algorithm) between the second camera and a third camera (i.e. numerous witness cameras) (p. 0189, 0200, 0239-0240). natural markers are automatically learned and computed in 3D by the system as the user scans the environment walking around. As soon as the user has started to do the survey, he has to also capture the registration object (see section 2.1.3.2) in order to automatically estimate the scale, origin and orientation of the world; and
wherein a positional relation between the second camera and the third camera is fixed (i.e. cameras can be attached to each other) (p. 0239-0240).
[Claim 17] Bolvin teaches The information processing apparatus according to claim 1,
wherein the circuitry is configured to estimate the offset information (i.e. 3D offset) by performing calibration based on a plurality of relative poses (i.e. determines positional information based on both cameras location which is determined, the cameras can be located anywhere) of the first camera and the second camera (i.e. film and witness cameras) (p. 0189, 0239-0240).
Claim 19 is analyzed and interpreted as a method of claim 1.
Claim 20 recites “A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer” to perform the steps of claim 1. Bolvin inherently discloses “A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer” to perform the steps of claim 1.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bolvin et al. (US 2015/0084951) In view of Someya (US 11218662), and further in view of Grabli et al. (US 2020/0286284).
[Claim 12] Bolvin is silent regarding The information processing apparatus according to claim 1, wherein the at least one marker is a retroreflective material, and wherein the second camera is an infrared camera.
Grabli teaches The information processing apparatus according to claim 1, wherein the at least one marker is a retroreflective material (i.e. retroreflective markers), and wherein the second camera is an infrared camera (i.e. IR) (p. 0025, 0037).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided camera types and reflective markers as taught by Grabli to the system of Bolvin to generate content (p. 0025)
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bolvin et al. (US 2015/0084951) In view of Someya (US 11218662), and further in view of Drixler (US 2020/0238525), and further in view of Tsuzuki (US 2015/0104097)
[Claim 18] Bolvin is silent regarding The information processing apparatus according to claim 1, wherein the circuitry is configured to estimate the offset information by solving hand-eye calibration based on a seven-degrees-of-freedom parameter including
a six-degrees-of-freedom parameter relating to relative positions and orientations of the first camera and the second camera, and a single-degree-of-freedom parameter relating to scale invariance.
Drixler teaches Tsuzuki teaches the specific feature of:
wherein the circuitry is configured to estimate the offset information by solving hand-eye calibration based on a seven-degrees-of-freedom parameter (i.e. degrees of freedom) (p. 0022).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided hand eye calibration as taught by Drixler to the system of Bolvin to provide calibration using degrees of freedom (p. 0022).
Tsuzuki teaches the specific feature of:
a six-degrees-of-freedom parameter (i.e. seven degrees of freedom) relating to relative positions and orientations of the first camera and the second camera (i.e. first and second cameras), and a single-degree-of-freedom parameter relating to scale invariance (p. 0036).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have provided degrees of freedom as taught by Tsuzuki to the system of Bolvin to provide a solution for scale indetermination (p. 0036).
Conclusion
Claims 1-20 are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20240144523 A1 Du; Ruofei et al.
US 20230336679 A1 Derry; Robert et al.
US 20220201269 A1 Atwell; Paul C. et al.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MUSHFIKH I ALAM whose telephone number is (571)270-1710. The examiner can normally be reached 1:00PM-9:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nasser Goodarzi can be reached at 571-272-4195. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MUSHFIKH I. ALAM
Primary Examiner
Art Unit 2426
/MUSHFIKH I ALAM/ Primary Examiner, Art Unit 2426 3/19/2026