DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim limitation “means for” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The disclosure is devoid of any corresponding structure in the specification that performs the means for generating a feature map that indicates relationships between pixels of the two-dimensional panorama image, including identifying pixels that correspond to one or more objects depicted in the two-dimensional panorama image; means for generating a volumetric representation by reshaping the feature map into a three-dimensional spherical map using a machine learning model based on the feature map, including generating one or more three-dimensional representations of the one or more objects for incorporation into the volumetric representation; and means for presenting the volumetric representation for display in a user interface. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gausebeck et al., U.S. Patent Publication Number 2019/026958 A1.
Regarding claim 1, Gausebeck discloses a method comprising: receiving, by a processing device, a two-dimensional panorama image (paragraph 0032, receiving a panoramic image; panoramic image can be received as projected on the two-dimensional plane); generating, by the processing device, a feature map that indicates relationships between pixels of the two-dimensional panorama image, including identifying pixels that correspond to one or more objects depicted in the two-dimensional panorama image (paragraph 0062, 2D images and/or feature (e.g., pixels, super pixels, objects, etc.) included in the 2D images to generate an alignment between the 2D images and/or the feature included in the respected 2D images, Examiner interprets alignment between the 2D image and/or feature as a feature map); and generating, by the processing device, a volumetric representation by rearranging the pixels indicated by the feature map into a three-dimensional spherical map using a machine learning model based on the feature map, including generating one or more three-dimensional representations of the one or more objects for incorporation into the volumetric representation (paragraph 0062, 3D model generation component further employ the alignment between the 2D image data and/or the associated 3D data to generate reconstructed representations or 3D models of objects or environments represented in the 2D image data).
Regarding claim 2, Gausebeck discloses wherein the two-dimensional panorama image is a surface of a sphere and depicts an indoor environment (paragraph 0062, 2D images captured of an interior of a home).
Regarding claim 3, Gausebeck discloses further comprising: receiving an input specifying a three-dimensional location relative to the volumetric representation to position a virtual three-dimensional object (paragraph 0088, selecting a location on the representation of the 3D model for viewing); inserting the virtual three-dimensional object at the three-dimensional location relative to the volumetric representation for display in a user interface (paragraph 0218, integrating virtual digital data objects on or within a live view of an environment); and presenting, by the processing device, the volumetric representation, including the virtual three-dimensional object, for display in the user interface (paragraph 0225, employ the derived 3D/depth data for an environment based on a current location and perspective of the view to the environment to facilitate user interaction with virtual AR data objects that are spatially integrated with the environment; paragraph 0051, determine a position for integrating a (virtual) graphical data object on or within a representation of the object or environment viewed on or through the display based on the current perspective and the 3D data).
Regarding claim 4, Gausebeck discloses wherein the machine learning model is trained on multiple two-dimensional panorama images (paragraph 0031, employing one or more machine learning models configured to predict 3D data from 2D images to derive 3D data for the 2D using deep learning techniques).
Regarding claim 5, Gausebeck discloses wherein the machine learning model is trained on random camera views of a training volumetric representation (paragraph 0099, input data comprises a 2D image with a field-of-view exceeding a defined threshold so as to classify it as a panoramic image (e.g., 120°, 150°, 180°, 250°, 350°, 359° etc.), which Examiner interprets as random camera views).
Regarding claim 6, Gausebeck discloses further comprising determining depicted depths of the pixels of the two-dimensional panorama image and incorporating the depicted depths into the feature map (paragraph 0032, deriving the 3D data based on wrapping around the panoramic image as projected on the 2D plane; )
Regarding claim 7, Gausebeck discloses further comprising tri-linearly interpolating points from the three-dimensional spherical map onto the volumetric representation (paragraph 0104, the output of a preceding convolutional layer can be interpolated and then used as input to the next subsequent or downstream layer).
Regarding claim 8, Gausebeck discloses wherein the three-dimensional spherical map is a concentric tri-sphere representation (paragraph 0070, 3D models described herein can include data representing positions, geometric shapes, curved surfaces, and the like. For example, a 3D model can include a collection of points represented by 3D coordinates, such as points in a 3D Euclidean space. The collection of points can be associated with each other (e.g. connected) by geometric entities, which Examiner interprets as a concentric tri-sphere representation in that the concentric tri-sphere representation would be comprised of a geometric shape, and connected by a geometric entity).
Regarding claim 9, Gausebeck discloses wherein pixels of the volumetric representation convey information about lighting, shadows, and reflections related to multiple viewpoints of content of the two-dimensional panorama image (paragraph 0123, 2D image, information regarding capture parameters of the capture device that generated the 2D image (e.g., focal length, resolution, lens distortion, lighting, other image metadata, etc.), actual depth data associated with the 2D image captured by a 3D sensor (e.g., 3D capture hardware), depth data derived for a 2D image using stereo image processing).
Regarding claim 10, it is rejected based upon similar rational as above claim 1. Gausebeck further discloses a non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations (paragraph 0061). Gausebeck further discloses translating the three-dimensional spherical map into a volumetric representation by decoding and upsampling the three-dimensional spherical map (paragraph 0041, alters one or more characteristics of the 2D image to transform the image into a pre-processed in accordance with a standard representation format (which Examiner interprets as decoding); paragraph 0069, models use multi-scale neural networks to separate overall scale prediction from prediction of the fine details. Some models refine the results by incorporating fully-connected layers, adding conditional random field (CRF) elements to the network, or predicting additional outputs such as normal vectors and combining those with the initial depth predictions to produce refined depth predictions (which Examiner interprets as upsampling in that elements are being added to the depth predications); and displaying the volumetric representation in a user interface (paragraph 0054, “3D model,” “3D object,” “3D reconstruction,” “3D image,” “3D representation,” “3D rendering,” “3D construct,” and the like are employed interchangeably throughout, unless context warrants particular distinctions among the terms. It should be appreciated that such terms can refer to data representing an object, space, scene, and the like in three dimensions, which may or may not be displayed on an interface).
Regarding claims 11-16, they are rejected based upon similar rational as above claims 2-6 and 9 respectively.
Regarding claim 17, Gausebeck further discloses a system (FIG. 1).
Regarding claims 18-20, they are rejected based upon similar rational as above claims 2, 6 and 9 respectively.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure..
Xu et al., U.S. Patent Publication Number 2024/0331117 A1
Xu discloses paragraph 0047, receiving the metadata of the two-dimensional panoramic projection image; paragraph 0156, a feature of the pixel in each region may be adaptively adapted.
Min et al., U.S. Patent Publication Number 2023/0125295 A1
Min discloses FIG. 2A, panorama images acquired; paragraph 0018, determine shapes or rooms or other areas and location of passages connecting rooms or other areas for some or all of those panorama images; paragraph 0023, 2D point cloud is supplied to a neural network of the IFPLMM system that has been trained to assign features to a floor plan’s 2D point cloud; paragraph 0018, generate other mapping information for building (e.g., a 3D model of the building interior and/or exterior), such as by using the inter-room passage information and other information to determine relative positions of the associated shapes of rooms or other areas.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Motilewa Good-Johnson whose telephone number is (571)272-7658. The examiner can normally be reached Monday - Friday 6am-2:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MOTILEWA . GOOD JOHNSON
Primary Examiner
Art Unit 2616
/MOTILEWA GOOD-JOHNSON/Primary Examiner, Art Unit 2619
Prosecution Insights