DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 5/16/2024 and 9/23/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shen, Weichao, et al. "PanoViT: Vision transformer for room layout estimation from a single panoramic image." arXiv preprint arXiv:2212.12156 (2022) hereinafter referred to as (Shen) in view of Li, Yuyan, et al. "Omnifusion: 360 monocular depth estimation via geometry-aware fusion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022 hereinafter referred to as (Li) in view of Cohen (U.S. Patent Pub. No 2017/0011488).
Regarding Claim 1, Shen teaches a method for processing a panorama image dataset, comprising:
generating a plurality of data panels from the panorama image dataset (Section 3.3: For panoramas, we sample square patches uniformly across the entire image. Given a panoramic image of size Cp ×Wp×Hp, we reshape it into N patches;)
executing a first neural network to process the plurality of data panels to generate a set of embeddings representing geometric features of the plurality of data panels (Section 3.3: We project different types of patches into patch embeddings through different embedding operations… we utilize a height compression module to calculate the patch embeddings of patches sampled from multi-scale feature maps… For patches sampled from the panoramic image, we use a simple multilayer perceptron to calculate their patch embeddings;)
executing a network to process the plurality of data panels and the set of embeddings to generate a plurality of mapping panels; and (Section 3.3: All the patch embeddings are attached with a recurrent position embedding to predict the room layout feature vector with a vision transformer backbone; All patch embeddings are concatenated together as the input of the vision transformer encoder; we combine the feature vector of all the patches sampled from the multi-scale feature maps as the outputted room layout feature vector)
Shen does not explicitly disclose a method for processing a panorama image dataset by a computing circuitry;
executing a second neural network to process the plurality of data panels and the set of embeddings to generate a plurality of mapping panels; and
fusing the plurality of mapping panels into a mapping dataset of the panorama image dataset.
Li is in the same field of art of image analysis. Further, Li teaches executing a first neural network to process the plurality of data panels to generate a set of embeddings representing geometric features of the plurality of data panels (Section 3.2: To compensate for the differences between patch-wise image features, we introduce a geometric embedding network (see Figure 2) to provide additional geometric information; The geometric embedding network consists of two layers of (Multi-Layer Perceptron) MLPs)
executing a second neural network to process the plurality of data panels and the set of embeddings to generate a plurality of mapping panels; and (Fig. 2; Section 3.3: we leverage the transformer architecture to aggregate information from the patches in a global fashion. The global aggregation is expected to improve the consistency of depth estimations from patches)
fusing the plurality of mapping panels into a mapping dataset of the panorama image dataset (Fig. 2; Section 3.4: The merged depth is then computed as a weighted average of all patch depth predictions with confidence scores used as weights.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Shen by using a transformer network to generate mapping panels then fusing them together that is taught by Li; thus, one of ordinary skilled in the art would be motivated to combine the references to improve scalability to high resolution inputs (Li Conclusion).
Cohen is in the same field of art of image analysis. Further, Cohen teaches
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Shen by using computing circuitry to process a panoramic image that is taught by Cohen; thus, one of ordinary skilled in the art would be motivated to combine the references to reduce computationally intensive activies (Cohen ¶2).
Regarding Claim 2, Shen in view of Li in view of Cohen discloses the method of claim 1, wherein the mapping dataset comprises one of a depth map, a layout map, or a semantic map corresponding to the panorama image dataset (Li, Fig. 2; Section 3.4: The merged depth is then computed as a weighted average of all patch depth predictions with confidence scores used as weights.)
The reasons for combining Shen and Li and Cohen are similar to that stated in the rejection of claim 1.
Regarding claim 15, claim 15 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Cohen further teaching on: An apparatus for processing a panorama image dataset, the apparatus comprising a memory for storing computer instructions and at least one processor for executing the computer instructions to (¶59 a processor 20, an imaging sensor 30, memory 40 and a position sensor 50. The processor 20 may be any suitable programmable control device and may control the operation of many functions... The memory 40 may include one or more storage mediums tangibly recording image data and program instructions)
Regarding claim 20, claim 20 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Cohen further teaching on: A non-transitory computer readable medium for storing instructions, the instructions, when executed by at least one processor (¶40 the present disclosure provides a computer program product implemented on a non-transitory computer usable medium having computer readable program code embodied therein to cause the computer to perform the image processing method and/or a panoramic image creation method)
Allowable Subject Matter
Claims 3-14 and 16-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 3 and 16 no prior art teaches, the panorama image dataset comprises a data array in two dimensions; and the each of the plurality of data panels comprises a subarray of the data array in an entirety of a first dimension of the two dimensions and a segment of a second dimension of the two dimensions.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
/JENNIFERMEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights