Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 19 June 2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7-9, 11, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Meier (EP 2770783) in view of Lee ("Object detection with sliding window in images including multiple similar objects," 2017 International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Korea (South), 2017, pp. 803-806) and further in view of Sulk (US 20160155011) and further in view of Gao (US 20190304105).
Regarding claim 11, Meier teaches a computer apparatus comprising:
A processor (Paragraph 94, host processor); and
A memory storing instructions that, when executed by the processor (Paragraph 16, hardwired logic or configurable logic which processes logical functions), configure the apparatus to:
Access a camera image (Paragraph 53, image capture) generated by an optical sensor (Paragraph 55, capture device) of an augmented reality (AR) device (Paragraph 2, augmented reality);
Access a query image (Paragraph 30, reference image) from a storage of the AR device (Paragraph 30, second capturing device, Paragraph 2, augmented reality);
Maintain the camera image at original, unscaled resolution (Paragraph 61, In order to extract features at a certain scale, either the sampling window can be scaled accordingly or the image is scaled before computing the response of the feature detector); Meier teaches the case where only the size of the sampling window is scaled and the original image is not scaled.
Scale a detector window of a feature detector program to a first scaled size of a first scaled detector window (Paragraph 61, the sampling window can be scaled);
Compare descriptors based on the extracted features from the camera image with descriptors based on the extracted features from the query image (Paragraph 61, In order to extract features at a certain scale, either the sampling window can be scaled accordingly or the image is scaled before computing the response of the feature detector; Paragraph 10, a feature descriptor is determined in order to enable the comparison and matching of features); and
Identify, using the feature detector program, the query image in the camera image based on the comparison of the descriptors (Paragraph 61, In order to extract features at a certain scale, either the sampling window can be scaled accordingly or the image is scaled before computing the response of the feature detector; Paragraph 10, a feature descriptor is determined in order to enable the comparison and matching of features).
While Meier fails to disclose the following, Lee teaches:
Maintaining the query image at original, unscaled resolution (Page 3, Paragraph 1, Rh denotes the height of a row in the bookshelf, and np denotes the number of book pages. The thickness of a book is estimated as 1 mm per every 10 pages.); Lee teaches maintaining the scale of the query image by matching the size of the sliding window to the query image.
Extract features from the unscaled query image by scanning the unscaled query image with a third unscaled detector window, wherein the third unscaled detector window is of a different size than each of the first scaled detector window and the second scaled detector window (Page 3, Paragraph 1, For example, if the heights of an image and a bookshelf with 5 rows are 1000 pixels and 2000 mm, the height of a window is computed as 100 pixels; Page 3, Paragraph 3, In a given image, the feature points are detected in each sliding window). Note: The size of the second and third unscaled detector windows are different than the size of the first scaled detector window because Meier teaches choosing the size of the scaled detector window.
Meier and Lee are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Meier by using Lee to maintain the scale of the query image and extract features from the unscaled query image using an unscaled detector. Doing so would allow for detecting predetermined desired features in the query image.
While the combination of Meier and Lee fails to disclose the following, Sulc teaches:
Extract features without generating an image pyramid of the camera image (Paragraph 4, A sliding window may be used to scan a large set of possible candidate windows. In this approach, a window is moved stepwise across the image in fixed increments so that a decision is computed for multiple overlapping windows. In practice, this approach uses windows of different sizes and aspect ratios to detect objects at multiple scales, with different shapes, and from different viewpoints. Consequently, millions of windows are tested per image. The computational cost is, therefore, one of the major impediments to practical detection systems; Paragraph 30-32, produce good product candidate regions (or “windows”)… The localization technique(s) may rely on keypoint detection, keypoint description, and keypoint matching to obtain an accurate localization of objects 12 in the image. For example, a set of candidate regions 44 that are predicted to be locations of objects 12 of interest, is identified… A feature extraction component 46 extracts a region descriptor 48 from each of the candidate regions).
Sulc and the combination of Meier and Lee are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Meier and Lee by using Sulc to use scaled detector windows to extract features without using an image pyramid. Doing so would allow for saving computation cost when determining the detector windows.
While the combination of Meier, Lee, and Sulc fails to disclose the following, Gao teaches:
Scale a detector window of a feature detector program to a plurality of different scaled sizes, the plurality of scaled sizes including a first scaled size of a first scaled detector window and at least a second scaled size of a second scaled detector window, the second scaled size being different from the first scaled size (Paragraph 59, first creates multiple search windows of different sizes for the current video frame such that these windows are all substantially centered on the same ROI determined for the previous video frame (step 502). In some embodiments, the sizes of the multiple search windows can be configured such that one or more of these search windows are larger than the first search window);
Extract features from the unscaled camera image by scanning the unscaled camera image with each of the first scaled detector window and the second scaled detector window, each scan a same single instance of the unscaled camera image without generating an image pyramid of the camera image, such that no scaled copy of the camera image is generated (Paragraph 59, first creates multiple search windows of different sizes for the current video frame such that these windows are all substantially centered on the same ROI determined for the previous video frame (step 502). In some embodiments, the sizes of the multiple search windows can be configured such that one or more of these search windows are larger than the first search window); Note: Gao teaches scanning a video frame, which has a fixed size.
Gao and the combination of Meier, Lee, and Sulc are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Meier, Lee, and Sulc by using Gao to use different size detector windows. Doing so would allow for anticipating the size of desired objects to be found within the frame (Gao, Paragraph 59).
Method claim 1 and CRM claim 20 correspond to apparatus claim 11. Therefore, claims 1 and 20 are rejected for the same reasons as used above.
Regarding claim 17, the combination of Meier, Lee, Sulc, and Gao teaches the computing apparatus of claim 11, wherein the camera image includes a low resolution image (Meier, Paragraph 4, the camera image may take a low resolution image).
Method claim 7 corresponds to apparatus claim 17. Therefore, claim 7 is rejected for the same reasons as used above.
Regarding claim 18, the combination of Meier, Lee, Sulc, and Gao teaches the computing apparatus of claim 11. While the combination as presented previously fails to disclose the following, Lee further teaches:
Wherein the feature detector program includes ORB (Oriented FAST and Rotated BRIEF) local feature detector (Page 3, Paragraph 3, an ORB detector is adopted).
Lee and the combination of Meier, Sulc, and Gao are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Meier, Sulc, and Gao by using Lee to use ORB. Doing so would consider trade-off between the accuracy and the speed in feature detection (Lee, Page 3, Paragraph 3).
Method claim 8 corresponds to apparatus claim 18. Therefore, claim 8 is rejected for the same reasons as used above.
Regarding claim 19, the combination of Meier, Lee, Sulc, and Gao teaches the computing apparatus of claim 11, wherein the feature detector program is configured to compare descriptors based on extracted features from the camera image using a scaled detector with descriptors based on extracted features from the query image using an unscaled detector (Meier, Paragraph 61, In order to extract features at a certain scale, either the sampling window can be scaled accordingly or the image is scaled before computing the response of the feature detector; Paragraph 10, a feature descriptor is determined in order to enable the comparison and matching of features).
Method claim 9 corresponds to apparatus claim 19. Therefore, claim 9 is rejected for the same reasons as used above.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Meier view of Lee and further in view of Sulc and further in view of Gao as applied to claims 1, 7-9, 11, and 17-20 above and further in view of Bean (US 9940692).
Regarding claim 15, the combination of Meier, Lee, Sulc, and Gao teaches the computing apparatus of claim 11. While it fails to disclose the following, Bean teaches:
Wherein the instructions further configure the apparatus to:
Access a virtual content item corresponding to the query image (col 1, Summary, identifying a first element of interest within the first image... associating a corresponding first AR content overlay for the first element of interest); and
Display the virtual content item in a display of the AR device (col 3, paragraph 4, Information (e.g., content overlays) associated with an element of interest can be displayed (e.g., in a callout, in a window, etc.) on an AR display).
Bean and the combination of Meier, Lee, Sulc, and Gao are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Meier, Lee, Sulc, and Gao by using Bean to determine an AR content overlay depending on a predetermined image. Doing so would provide users with AR overlays correlated to specific query images, customizing their AR experience.
Method claim 5 corresponds to apparatus claim 15. Therefore, claim 5 is rejected for the same reasons as used above.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Meier view of Lee and further in view of Sulc and further in view of Gao as applied to claims 1, 7-9, 11, and 17-20 above and further in view of Reed (US 20180284454).
Regarding claim 16, the combination of Meier, Lee, Sulc, and Gao teaches the computing apparatus of claim 11. While it fails to disclose the following, Reed teaches:
Wherein the instructions further configure the apparatus to: generate the camera image using a wide-angle lens coupled to the optical sensor (Paragraph 82, the optical component may be a wide angle lens).
Reed and the combination of Meier, Lee, Sulc, and Gao are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Meier, Lee, Sulc, and Gao by using Reed to incorporate a wide-angle camera. Doing so would allow the system to enlarge the field of view of the camera (Reed, paragraph 82).
Method claim 6 corresponds to apparatus claim 16. Therefore, claim 6 is rejected for the same reasons as used above.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Meier view of Lee and further in view of Sulc and further in view of Gao as applied to claims 1, 7-9, 11, and 17-20 above and further in view of Davidson (WO 2022254409).
Regarding claim 10, the combination of Meier, Lee, Sulc, and Gao teaches the method of claim 1. While it fails to disclose the following, Davidson teaches:
Wherein the query image includes marker data that indicates a pre-defined visual code (Paragraph 181, The reference features may be a quick response (QR) code or known exemplar or marker, which can provide processor 310 certain information).
Davidson and the combination of Meier, Lee, Sulc, and Gao are both considered to be analogous to the claimed invention because they are in the same field of augmented reality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Meier, Lee, Sulc, and Gao by using Davidson to a pre-defined visual code (QR code) as the query image. Doing so would allow the query image to be more easily identifiable in the camera image.
Response to Arguments
Applicant’s arguments with respect to claims 1, 11, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding claim 11, Applicant argues that Meier, Lee, and Sulc all do not teach a plurality of differently-scaled detector windows applied to a single unscaled camera image. However, Gao addresses this for the amended limitations as cited above. It would have been obvious to a person of ordinary skill in the art to combine this with the teachings of Meier, Lee, and Sulc to address the amended limitations of claim 11. Claims 1 and 20 are rejected for the same reasons as used above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SNIGDHA SINHA whose telephone number is (571)272-6618. The examiner can normally be reached Mon-Fri. 12pm-8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SNIGDHA SINHA/Examiner, Art Unit 2619
/JASON CHAN/Supervisory Patent Examiner, Art Unit 2619
Prosecution Insights