Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 21, 2025; January 30, 2025; July 06, 2026 and August 12, 2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5 and 6 are rejected under 35 U.S.C 103 as being unpatentable over Oi et al. Patent Application Publication No. JP-2008304268-A5 (hereinafter Oi) in view of Yalan ‘A Review of V-SLAM’ (hereinafter Yalan).
Regarding claim 1, Oi discloses an image information processing apparatus for receiving a series of pieces of image data obtained by a moving camera capturing an image in a three- dimensional space (Oi in [0047] discloses about moving camera and moving image (series of image), “an example of processing when the user (physical agent) holds the camera 101 and freely moves to capture a moving image and inputs the captured image to the image recognition processing unit 102 and the self-position detection unit 103”. Furthermore, Oi in [0036] discloses about 3D space, “the process of estimating the position of the feature point and the camera position in the three-dimensional space simultaneously every frame is performed”), in order to estimate posture information regarding the camera capturing each piece of image data and estimate map information regarding a captured target (Oi in [0002] discloses, “executes a mapping (environment map) around the agent ... and a computer program for executing an estimation process (localization) of an agent's position or posture, that is, an estimation of the position and posture of an agent in conjunction with an environment map creation process”), the image information processing apparatus comprising: a processor executing an estimation process of estimating the posture information regarding the camera at multiple points and the map information (Oi in [0048] discloses, “The self-position detection unit 103 corrects the position of the feature point and the camera position for each frame using the feature points in the video input from the camera 101, and is expressed in the world coordinate system determined by the self-position detection unit 103. The camera position (Cw) and the camera rotation matrix (Rw) as the estimated camera position and orientation information are output to the data construction unit 110” wherein determining the camera position and the camera rotation matrix equates to posture information for each frame (multiple points)), the processor recognizes an object in the three- dimensional space captured in each of the multiple pieces of image data (Oi in [0033] discloses about recognizing objects, “The image recognition processing unit 102 outputs a recognition result including position and orientation information in images of various objects (detection targets) photographed by the camera 101”. Furthermore, Oi in [0036] discloses about 3D space and [0047] discloses about multiple pieces of image data), so as to select the selection point by a predetermined method from over the recognized object (Oi in [0054] discloses, “identifies the identified object. The feature point position in the object image 311 corresponding to the feature point information registered as dictionary data corresponding to is determined. Assume that the feature point positions are four vertices α, β, γ, and δ of the rectangular parallelepiped as shown in the figure”) captured in common in the multiple pieces of image data (Oi in [0074] discloses about feature points associated with the recognized objects are reused across frames implies to captured in common, “The feature point P is a feature point position detected in the past frame. The coordinate conversion unit 403 calculates which position of the captured image 420 that is the current frame the position of the feature point P extracted from the past frame corresponds to, and provides it to the image recognition processing unit 102”).
Oi doesn’t disclose the following limitation as further recited in the claim.
Yalan discloses an optimization process of optimizing the estimated map information (Yalan in [Page – 605, Paragraph – 2 (Right)] discloses, “LSD SLAM contains three threads, camera pose tracking, depth estimation and global map optimization”. Furthermore, Yalan in [Page – 605, Paragraph – 2 (Right)] discloses, “LSD SLAM contains three threads, camera pose tracking, depth estimation and global map optimization”) by projecting a common selection point in the captured image in the three-dimensional space to multiple pieces of the captured image data to obtain a set of projection points as related points (Yalan in [Section – II(A), Paragraph – 1] discloses about common point
X
j
projected through multiple camera poses
T
i
. Furthermore, Yalan in [Section – II(A), Paragraph – 2] also discloses about projection, “As shown in Fig.(1), feature-based method extracts the feature points from the images and match the descriptors, then it will obtain the re projection error between corresponding feature points h11 and h21”), before comparing, in terms of luminance, the projection points with each other as the related points, wherein, in the optimization process, (Yalan in [Section – II(A), Paragraph – 2] discloses about comparing (h11 – h21) related points (corresponding feature points h11 and h21), “As shown in Fig.(1), feature-based method extracts the feature points from the images and match the descriptors, then it will obtain the re projection error between corresponding feature points h11 and h21 .... find the pixel positions through pixel gradient, and solves the optimal pose of camera through optimizing photometric error I (h11) − I (h21)”. Lastly, Yalan in [Section – II(C), Paragraph – 1] discloses about pixel brightness information (luminance)).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Yalan into the system of Oi because it would allow the system to output a more accurate map.
Summary of Citations (Yalan)
[Page – 604, Paragraph – 1 (Left)]; “The back-end solves the problem of optimizing the historical trajectory of vehicle”.
[Section – II(A), Paragraph – 1];
PNG
media_image1.png
596
1028
media_image1.png
Greyscale
[Section – II(A), Paragraph – 2]; “As shown in Fig.(1), feature-based method extracts the feature points from the images and match the descriptors, then it will obtain the re projection error between corresponding feature points h11 and h21 .... find the pixel positions through pixel gradient, and solves the optimal pose of camera through optimizing photometric error I (h11) − I (h21)”.
[Page – 605, Paragraph – 2 (Right)]; “LSD SLAM contains three threads, camera pose tracking, depth estimation and global map optimization”.
[Section – II(C), Paragraph – 1]; “The feature-based SLAM methods are sensitive to the richness of the environmental features and the image quality, such as blurred degree, image noise. The direct methods estimate the motion of camera directly according to pixel brightness information”.
Summary of Citations (Oi)
Paragraph [0002]; “executes a mapping (environment map) around the agent ... and a computer program for executing an estimation process (localization) of an agent's position or posture, that is, an estimation of the position and posture of an agent in conjunction with an environment map creation process”.
Paragraph [0033]; “The image recognition processing unit 102 outputs a recognition result including position and orientation information in images of various objects (detection targets) photographed by the camera 101”.
Paragraph [0036]; “Based on the change in the position of the local region (hereinafter referred to as feature point) in the image between the frames, the process of estimating the position of the feature point and the camera position in the three-dimensional space simultaneously every frame is performed”.
Paragraph [0047]; “an example of processing when the user (physical agent) holds the camera 101 and freely moves to capture a moving image and inputs the captured image to the image recognition processing unit 102 and the self-position detection unit 103”.
Paragraph [0048]; “The self-position detection unit 103 corrects the position of the feature point and the camera position for each frame using the feature points in the video input from the camera 101, and is expressed in the world coordinate system determined by the self-position detection unit 103. The camera position (Cw) and the camera rotation matrix (Rw) as the estimated camera position and orientation information are output to the data construction unit 110”.
Paragraph [0054]; “identifies the identified object. The feature point position in the object image 311 corresponding to the feature point information registered as dictionary data corresponding to is determined. Assume that the feature point positions are four vertices α, β, γ, and δ of the rectangular parallelepiped as shown in the figure”.
Paragraph [0074]; “The feature point P is a feature point position detected in the past frame. The coordinate conversion unit 403 calculates which position of the captured image 420 that is the current frame the position of the feature point P extracted from the past frame corresponds to, and provides it to the image recognition processing unit 102”.
Regarding claim 5, method claim 5 corresponds to apparatus claim 1. Therefore, the
rejection analysis and motivation to combine claim 1 is applicable to claim 5.
Regarding claim 6, is a non-transitory computer readable storage medium claim
corresponds to apparatus claim 1. Therefore, the rejection analysis of claim 1 is applied in claim 6.
Claims 2 and 3 are rejected under 35 U.S.C 103 as being unpatentable over Oi in view of Yalan and further in view of Zhou US Patent Application Publication No. US-20240029297-A1 (hereinafter Zhou).
Regarding claim 2, Oi in the combination discloses the image information processing apparatus according to claim 1, wherein the predetermined method for selecting the selection point involves selecting the selection point from a region of the recognized object (Oi in [0007] discloses, “the image recognition processing unit executes a process of identifying a position of a feature point of an object included in an image acquired by the camera and outputting it to the data construction unit”).
Oi and Yalan in the combination don’t disclose the following limitation as further recited in the claim.
Zhou discloses the region corresponding to an edge of the object (Zhou in [0101] discloses, “The feature points refer to representative points for a local part of the image, and may reflect local features of the image. The feature points are generally extracted from a boundary region having a rich texture”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Zhou into the system of Oi in view of Yalan because it would allow the system to reduce computation and make the process more efficient by selecting only certain point on object edges.
Summary of Citations (Zhou)
Paragraph [0101]; “The feature points refer to representative points for a local part of the image, and may reflect local features of the image. The feature points are generally extracted from a boundary region having a rich texture”.
Summary of Citations (Oi)
Paragraph [0007]; “the image recognition processing unit executes a process of identifying a position of a feature point of an object included in an image acquired by the camera and outputting it to the data construction unit”.
Regarding claim 3, Oi in the combination discloses the image information processing apparatus according to claim 1.
Oi and Yalan in the combination don’t disclose the following limitation as further recited in the claim.
Zhou discloses the predetermined method for selecting the selection point involves selecting the selection point from a region of the recognized object according to complexity of texture (Zhou in [0101] discloses, “The feature points refer to representative points for a local part of the image, and may reflect local features of the image. The feature points are generally extracted from a boundary region having a rich texture”. Furthermore, Zhou in [0078] also discloses point selection).
Summary of Citations (Zhou)
Paragraph [0078]; “pixel points in a low flatness region (which usually refers to a region with large texture variations) in the current image frame are selected”.
Paragraph [0101]; “The feature points refer to representative points for a local part of the image, and may reflect local features of the image. The feature points are generally extracted from a boundary region having a rich texture”.
Claim 4 is rejected under 35 U.S.C 103 as being unpatentable over Oi in view of Yalan and further in view of Nan ‘D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry’ (hereinafter Nan).
Regarding claim 4, Oi in the combination discloses the image information processing apparatus according to claim 1.
Oi and Yalan in the combination don’t disclose the following limitation as further recited in the claim.
Nan discloses the processor further performs a second optimization process using a result of the optimization process and a coordinate error (Nan in [Page – 5, Paragraph – 2] discloses about front-end and optimization, “After the optimization is finished, we marginalize the last frame and the factor graph will be used for the front-end tracking of the following frame”. Nan in [Page – 5, Paragraph – 3&4] discloses about tracking front end is used to initialize (using result of the optimization process) photometric bundle adjustment backend (second optimization process) and twist coordinate. Nan in [Page – 5, Paragraph – 3&4] also discloses,
E
t
o
t
a
l
is minimized using the Gauss-Newton method (second optimization) wherein
E
p
o
s
e
(coordinate or pose) is part or
E
t
o
t
a
l
.
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Nan into the system of Oi in view Yalan because it would allow the system to improve the camera posture estimation.
Summary of Citations (Zhou)
[Page – 5, Paragraph – 2]; “After the optimization is finished, we marginalize the last frame and the factor graph will be used for the front-end tracking of the following frame”.
[Page – 5, Paragraph – 3&4];
PNG
media_image2.png
1386
1066
media_image2.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to ZAID MUHAMMAD SALEH whose telephone number is (703)756-1684.
The examiner can normally be reached M-F 8 am - 5 pm ET. Examiner interviews are available
via telephone, in-person, and video conferencing using a USPTO supplied web-based
collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO
Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.If attempts to
reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be
reached on (571)272-7332. The fax phone number for the organization where this application or
proceeding is assigned is 571-273-8300. Information regarding the status of published or
unpublished applications may be obtained from Patent Center. Unpublished application
information in Patent Center is available to registered users. To file and manage patent
submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center
and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For
additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative, call 800-786
9199 (IN USA OR CANADA) or 571-272-1000.
/ZAID MUHAMMAD SALEH/
Examiner, Art Unit 2668
09/19/2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights