DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Preliminary Amendment
This Office Action is responsive to a preliminary amendment filed on 08/19/2024. Claims 1-8 are pending in the instant application. Claims 1, 7 and 8 are independent. Applicant submits no new matter has been added. An Office Action on the merits follows her below.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/19/2024, 08/20/2024, 08/20/2024 and 12/01/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Appropriate correction is requested.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “image acquiring unit”; “a point cloud data acquiring unit”; and “a controller” at least in independent claims 1.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recites sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
35 U.S.C. 101 requires that a claimed invention must fall within one of the four eligible categories of invention (i.e. process, machine, manufacture, or composition of matter) and must not be directed to subject matter encompassing a judicially recognized exception as interpreted by the courts. MPEP 2106. The four eligible categories of invention include: (1) process which is an act, or a series of acts or steps, (2) machine which is an concrete thing, consisting of parts, or of certain devices and combination of devices, (3) manufacture which is an article produced from raw or prepared materials by giving to these materials new forms, qualities, properties, or combinations, whether by hand labor or by machinery, and (4) composition of matter which is all compositions of two or more substances and all composite articles, whether they be the results of chemical union, or of mechanical mixture, or whether they be gases, fluids, powders or solids. MPEP 2106(I).
Claim 8 is rejected under 35 U.S.C. 101 as not falling within one of the four statutory categories of invention because the claimed invention is directed to computer program per se. See MPEP 2106(I). A claim directed toward a “non-transitory computer-readable medium” having the program encoded thereon establishes a sufficient functional relationship between the program and a computer so as to remove it from the realm of “program per se”. MPEP 2111.05(III). Hence, adding the limitation of “stored on a non-transitory computer-readable medium” could resolve this issue. Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 5-7 and 8 are rejected under 35 U.S.C. 102(a)(1) and/or (a)(2) as being anticipated by DeTone (US 20190005670 A1).
Regarding Claim 1: (Original) DeTone discloses an information processing device (Refer to para [004]; “Embodiments of the present invention enable the accurate detection of user/device movement by analyzing the images captured by a device worn by the user, thereby improving the accuracy of the displayed virtual content. Although the present invention may be described in reference to an AR device, the disclosure is applicable to a variety of applications in computer vision and image display systems.”) comprising: an image acquiring unit configured to acquire an image from an image-capturing device (Refer to para [005 and 007]; “In some embodiments, the first image was captured by a first camera at a first instant in time. In some embodiments, the second image was captured by the first camera at a second instant in time after the first instant in time.”) a point cloud data acquiring unit configured to acquire point cloud data representing a distance distribution from a distance-measuring device (Refer to para [005]; “The method may also include generating a first point cloud based on the first image and a second point cloud based on the second image. The method may further include providing the first point cloud and the second point cloud to a neural network. The method may further include generating, by the neural network, the homography based on the first point cloud and the second point cloud. In some embodiments, the first point cloud and the second point cloud are two-dimensional (2D) point clouds. In some embodiments, the first image was captured by a first camera at a first instant in time. In some embodiments, the second image was captured by the first camera at a second instant in time after the first instant in time. In some embodiments, the first point cloud and the second point cloud are generated using a first neural network, and the neural network is a second neural network.”) and a controller (Refer to para [047]; “FIG. 8 illustrates a method 800 for computing a homography based on two images. Steps of method 800 may be performed in a different order than that shown, and one or more steps of method 800 may be omitted during performance of method 800. One or more steps of method 800 may be performed and/or initiated by a processor configured to execute instructions contained in a non-transitory computer-readable medium.”) configured to, based on a first image and a second image acquired from the image acquiring unit, the second image containing an identical subject to the first image (Refer to para [007]; “The AR device may also include a processor communicatively coupled to the camera and configured to perform operations including: receiving, from the camera, a first image based on a first camera pose and a second image based on a second camera pose, generating a first point cloud based on the first image and a second point cloud based on the second image, providing the first point cloud and the second point cloud to a neural network, and generating, by the neural network, a homography based on the first point cloud and the second point cloud.”) acquire information representing a correspondence relationship between positions in the first image and the second image (Refer to para [051]; “At step 808, a homography is generated based on the first point cloud and the second point cloud using the second neural network. In some embodiments, the generated homography comprises a matrix (e.g., 3×3) from which a relative rotation and a relative translation (i.e., a relative pose) between the first camera pose and the second camera pose may be extracted.”) and based on the information representing the correspondence relationship, create associations between two sets of point cloud data acquired from the point cloud data acquiring unit, the two sets of point cloud data consisting of first point cloud data in which a capturing range of the first image and a measurement range of the distance distribution at least partially overlap and second point cloud data in which a capturing range of the second image and a measurement range of the distance distribution at least partially overlap (Refer to para [055 and 064]; “At step 906, the 3D trajectory may be sampled to obtain a particular first camera pose and a particular second camera pose. In some embodiments, the plurality of points are at least partially viewable from the particular first camera pose and the particular second camera pose. For example, the obtained camera poses may be restricted to those camera poses that view at least 25%, 50%, 75%, or 100% of the plurality of points. If a camera pose does not meet a predetermined threshold (e.g., that at least 50% of the plurality of points are viewable), then the camera pose is discarded and the 3D trajectory is resampled to obtain another camera pose. In some embodiments, the obtained camera poses are restricted to have at least some threshold of visual overlap (e.g., 30%) with each other. In some embodiments, the visual overlap may correspond to the percentage of points of the plurality of points that are viewable by both the particular first camera pose and the particular second camera pose. In other embodiments, the visual overlap may be calculated based on the shared field of views between the obtained poses.”).
Regarding Claim 5: (Currently Amended) DeTone discloses the controller is configured to calculate the information representing the correspondence relationship (Refer to para [060]; “At step 916, the neural network is modified based on the comparison between the particular homography and the ground-truth homography performed in step 914 by, for example, adjusting one or more weights or coefficients of the neural network. In some embodiments, the neural network may be modified based on the calculated difference between the homographies (i.e., the error signal) such that a larger error signal causes a greater modification to neural network. In general, modifying the neural network causes the neural network to become more accurate thereby decreasing the difference between the particular homography and the ground-truth homography.”) by performing matching of feature points contained in the first image and the second image (Refer to para [030]; “In some embodiments, the networks are able to solve the following problems: detecting robust 2D locations in noisy images, computing the relative pose between two images, and relocalization. Unlike conventional approaches, which rely heavily on both engineered feature descriptors (ORB or SIFT), embodiments of the present invention may not associate descriptors with individual points in images. Unlike these conventional feature-based SLAM systems, relative pose estimation may be performed in a descriptor-less fashion. Embeddings that may be similar to global image-wide descriptors may also be used. The embeddings may be engineered to be pseudo homographic invariant. By design, two images that are related by a homography may be close on a given manifold.”).
Regarding Claim 6: (Original) DeTone discloses the controller is configured to perform matching of the feature points using a homographic transformation (Refer to para [041 and 042]; “In some embodiments, one goal of the embedding network may be to associate a global 128 dimensional descriptor with the input image. In some embodiments, it is desirable that the embedding is homographic invariant. For example, two images that are related by a homography should have the same embedding vector, and two images that are not depicting the same scene content (and thus not the same plane) should have different embedding vectors.”).
Regarding Claim 7: (Original) DeTone discloses an information processing method (Refer to para [004]; “Embodiments of the present invention enable the accurate detection of user/device movement by analyzing the images captured by a device worn by the user, thereby improving the accuracy of the displayed virtual content. Although the present invention may be described in reference to an AR device, the disclosure is applicable to a variety of applications in computer vision and image display systems.”) comprising: acquiring a first image and a second image containing an identical subject to the first image (Refer to para [005 and 007]; “In some embodiments, the first image was captured by a first camera at a first instant in time. In some embodiments, the second image was captured by the first camera at a second instant in time after the first instant in time.”) acquiring two sets of point cloud data representing a distance distribution (Refer to para [005]; “The method may also include generating a first point cloud based on the first image and a second point cloud based on the second image. The method may further include providing the first point cloud and the second point cloud to a neural network. The method may further include generating, by the neural network, the homography based on the first point cloud and the second point cloud. In some embodiments, the first point cloud and the second point cloud are two-dimensional (2D) point clouds. In some embodiments, the first image was captured by a first camera at a first instant in time. In some embodiments, the second image was captured by the first camera at a second instant in time after the first instant in time. In some embodiments, the first point cloud and the second point cloud are generated using a first neural network, and the neural network is a second neural network.”) the two sets of point cloud data consisting of first point cloud data in which a capturing range of the first image and a measurement range of the distance distribution at least partially overlap and second point cloud data in which a capturing range of the second image and a measurement range of the distance distribution at least partially overlap (Refer to para [007, 055 and 064]; “The AR device may also include a processor communicatively coupled to the camera and configured to perform operations including: receiving, from the camera, a first image based on a first camera pose and a second image based on a second camera pose, generating a first point cloud based on the first image and a second point cloud based on the second image, providing the first point cloud and the second point cloud to a neural network, and generating, by the neural network, a homography based on the first point cloud and the second point cloud.”) acquiring information representing a correspondence relationship between positions in the first image and the second image based on the first image and the second image (Refer to para [051]; “At step 808, a homography is generated based on the first point cloud and the second point cloud using the second neural network. In some embodiments, the generated homography comprises a matrix (e.g., 3×3) from which a relative rotation and a relative translation (i.e., a relative pose) between the first camera pose and the second camera pose may be extracted.”) and creating associations between the first point cloud data and the second point cloud data based on the information representing the correspondence relationship (Refer to para [055 and 064]; “At step 906, the 3D trajectory may be sampled to obtain a particular first camera pose and a particular second camera pose. In some embodiments, the plurality of points are at least partially viewable from the particular first camera pose and the particular second camera pose. For example, the obtained camera poses may be restricted to those camera poses that view at least 25%, 50%, 75%, or 100% of the plurality of points. If a camera pose does not meet a predetermined threshold (e.g., that at least 50% of the plurality of points are viewable), then the camera pose is discarded and the 3D trajectory is resampled to obtain another camera pose. In some embodiments, the obtained camera poses are restricted to have at least some threshold of visual overlap (e.g., 30%) with each other. In some embodiments, the visual overlap may correspond to the percentage of points of the plurality of points that are viewable by both the particular first camera pose and the particular second camera pose. In other embodiments, the visual overlap may be calculated based on the shared field of views between the obtained poses.”).
Regarding Claim 8: (Original) DeTone discloses a program for causing a computer to execute processing (Refer to para [004 and 047]; “Embodiments of the present invention enable the accurate detection of user/device movement by analyzing the images captured by a device worn by the user, thereby improving the accuracy of the displayed virtual content. Although the present invention may be described in reference to an AR device, the disclosure is applicable to a variety of applications in computer vision and image display systems.” “One or more steps of method 800 may be performed and/or initiated by a processor configured to execute instructions contained in a non-transitory computer-readable medium.”) comprising: acquiring a first image and a second image containing an identical subject to the first image (Refer to para [005 and 007]; “In some embodiments, the first image was captured by a first camera at a first instant in time. In some embodiments, the second image was captured by the first camera at a second instant in time after the first instant in time.”) acquiring two sets of point cloud data representing a distance distribution (Refer to para [005]; “The method may also include generating a first point cloud based on the first image and a second point cloud based on the second image. The method may further include providing the first point cloud and the second point cloud to a neural network. The method may further include generating, by the neural network, the homography based on the first point cloud and the second point cloud. In some embodiments, the first point cloud and the second point cloud are two-dimensional (2D) point clouds. In some embodiments, the first image was captured by a first camera at a first instant in time. In some embodiments, the second image was captured by the first camera at a second instant in time after the first instant in time. In some embodiments, the first point cloud and the second point cloud are generated using a first neural network, and the neural network is a second neural network.”) the two sets of point cloud data consisting of first point cloud data in which a capturing range of the first image and a measurement range of the distance distribution at least partially overlap and second point cloud data in which a capturing range of the second image and a measurement range of the distance distribution at least partially overlap (Refer to para [007, 055 and 064]; “The AR device may also include a processor communicatively coupled to the camera and configured to perform operations including: receiving, from the camera, a first image based on a first camera pose and a second image based on a second camera pose, generating a first point cloud based on the first image and a second point cloud based on the second image, providing the first point cloud and the second point cloud to a neural network, and generating, by the neural network, a homography based on the first point cloud and the second point cloud.”) acquiring information representing a correspondence relationship between positions in the first image and the second image based on the first image and the second image (Refer to para [051]; “At step 808, a homography is generated based on the first point cloud and the second point cloud using the second neural network. In some embodiments, the generated homography comprises a matrix (e.g., 3×3) from which a relative rotation and a relative translation (i.e., a relative pose) between the first camera pose and the second camera pose may be extracted.”) and creating associations between the first point cloud data and the second point cloud data based on the information representing the correspondence relationship (Refer to para [055 and 064]; “At step 906, the 3D trajectory may be sampled to obtain a particular first camera pose and a particular second camera pose. In some embodiments, the plurality of points are at least partially viewable from the particular first camera pose and the particular second camera pose. For example, the obtained camera poses may be restricted to those camera poses that view at least 25%, 50%, 75%, or 100% of the plurality of points. If a camera pose does not meet a predetermined threshold (e.g., that at least 50% of the plurality of points are viewable), then the camera pose is discarded and the 3D trajectory is resampled to obtain another camera pose. In some embodiments, the obtained camera poses are restricted to have at least some threshold of visual overlap (e.g., 30%) with each other. In some embodiments, the visual overlap may correspond to the percentage of points of the plurality of points that are viewable by both the particular first camera pose and the particular second camera pose. In other embodiments, the visual overlap may be calculated based on the shared field of views between the obtained poses.”).
Allowable Subject Matter
Claims 2-4 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The prior art either singly or in combination does not teach, disclose or suggest at least the following claim limitation(s): “… the controller estimates an amount of rotation and an amount of movement of the image-capturing device or the distance-measuring device based on the associations between the first point cloud data and the second point cloud data.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIA M THOMAS whose telephone number is (571)270-1583. The examiner can normally be reached M-Th 8:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen (Steve) Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MIA M. THOMAS
Primary Examiner
Art Unit 2665
/MIA M THOMAS/Primary Examiner
Art Unit 2665
Prosecution Insights