DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
The Office Action has been made issued in response to amendment filed February 05, 2026. Claims 1-20 are pending. Applicant’s arguments have been carefully and respectfully considered in light of the instant amendment, and are not persuasive. Accordingly, this action has been made FINAL.
Response to Arguments
Claim Objection
Applicant has amended claims 1 and 11 as suggested by the Examiner. Therefore, the objection has been removed.
Claim Interpretation
Applicant has failed to address the 112 (f) interpretation of claim 11. The Examiner suggest Applicant acknowledge the 112(f) interpretation of the claim elements in the next Response.
Claim Rejections – 35 USC section § 102/103
Applicant's arguments with respect to claim 1-2,5-6,8-12,15-16 and 18-20 have been considered but are moot in view of the new ground(s) of rejection.
Claim Objections
Claims 19 is objected to because of the following informalities:
In claim 19 recites the term “A non-transitory computer-readable medium storing instructions that, when executed, cause;” should be changed to, “A non-transitory computer-readable medium storing instructions that, when executed, by a processor, causes the processor to execute” in order to avoid prevent a rejection under 35 U.S.C.101.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claim 11 recites limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
· Claim 11; recites the limitation, “each display element configured to….” [Line 3].
· Claim 11; recites the limitation, “the one or more imaging devices configured to ….” [Line 5,].
· Claim 11; recites the limitation, “a depth camera assembly configured to….” [Line 7].
· Claim 11; recites the limitation, “an object detection module including ……. cause the headset to: detect an object….” [Line 9-12].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claim 11:
(i) “display element” (Fig. 1, #120. Paragraph [0023]- display element is described as a display element 120 may be a waveguide display. A waveguide display includes a light source (e.g., a two-dimensional source, one or more line sources, one or more point sources, etc.) and one or more waveguides. Fig. 1, illustrates the display element as a black box. (Wherein the display element do have sufficient structure associated with it a waveguide light source.).
(ii) “imaging devices” (Fig. 1, #130. Paragraph [0036]- imaging devices is described as may include a passive camera assembly (PCA) that generates color image data. The PCA may include one or more RGB cameras that capture images of some or all of the local area. In some embodiments, some or all of the imaging devices 130 of the DCA may also function as the PCA. Fig. 1, illustrates the imaging devices as a black box. (Wherein the imaging devices do have sufficient structure associated with it, cameras.).
(iii) “depth camera assembly” (Fig. 6, #645. Paragraph [0021 and 0026]- depth camera assembly is described as a depth camera assembly (DCA). The DCA includes one or more imaging devices 130 and a DCA controller Fig. 6, illustrates the depth camera assembly as a black box. (Wherein the depth camera assembly do have sufficient structure associated with it cameras and controller.).
(iv) “an object detection module” (Fig. 2, #200. Paragraph [0039]- an object detection module is described as includes a processor and one or more non- transitory computer readable storage media. Fig. 2, illustrates the object detection module as a black box. (Wherein the object detection module do have sufficient structure associated with it processor and memory.).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 11, 15 and 19 are rejected under 35 U.S.C. 103 as being anticipated by Hall et al (US Patent No.: 10972715) in view of Khan et al (NPL titled: Efficient and Scalable Object Localization in 3D on Mobile Device).
Regarding independent claim 1, Hall discloses a method (method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest,– see col 1, lines 61-62) comprising:
capturing, at a headset worn (headset – see Fig 1) by a user (user - see col 3, lines 47), two-dimensional images of a local area surrounding the headset (an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” – see col 1, lines 61-64) by one or more imaging devices (150, passive camera assembly – see Fig 2) included in the headset (see Fig 2);
detecting an object in the local area from two-dimensional image of the local are captured by an imaging device of the one or more image devices(determine positions of an object – see col 9, lines 11-12);
determining a local area model of the local are from depth information (The DCA 140 generates depth image data of a local area, such as a room. Depth image data includes pixel values defining distance from the DCA 140,– see col 5, lines 28-30) generated by one or more depth sensors (140, depth camera – see Fig 2) included in the headset (see Fig 2), the local area model comprising a three-dimensional reconstruction of the local area to obtain a three-dimensional local area model of the local area (DCA 140 also provides a mapping of locations captured in the depth image data, such as a three-dimensional mapping of locations captured in the depth image data - see col 5, lines 30-33); and
storing the position of the object in the three-dimensional local area (an example of a virtual model 600 describing local areas and parameters describing configuration of the local areas. The virtual model 600 may be stored in the virtual model database 505 of the mapping server 130. The virtual model 600 may represent geographic information storage area in the virtual model database 505 that stores geographically tied triplets of information (i.e., a local area identifier (ID) 605, a local area configuration ID 610, and a set of c parameters 615) for various local areas surrounding one or more headsets 110 - see col 18, lines 63-67 through col 19, lines 1-5) in association with information identifying the object (the virtual model 600 includes a listing of possible local areas S1, S2, . . . , Sn, each identified by a unique local area ID 605 – see col 19, lines 6-9); however, Hall does not explicitly teach determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
However, Khan explicitly teaches determining a bounding box for the object (see Fig. 1), the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object (estimated using 2D bounding box coordinate – see section 1, [p][003] and Fig. 1); and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device ([o]nce we obtain the 2D bounding box for the detected object in the image scene, we estimate the 3D cuboid for the object using the method – see section 3.2.3).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Khan of determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
Wherein having Hall`s determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
The motivation behind the modification would have been to create a virtual 3D space for enhancing human experience with reduction in power consumption hence portability since both Hall and Khan are methods for camera localization system and localization mapping of the real world. Wherein Hall selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Khan proposes an object localization solution in Three-Dimension (3D) for mobile devices using a 2D object detection Convolutional Neural Network (CNN) model with Augmented Reality (AR) technologies to recognize objects in the environment and determine their real-world coordinates (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Khan et al (NPL titled: Efficient and Scalable Object Localization in 3D on Mobile Device) Abstract).
Regarding to claim 5, Hall in view of Gupta teaches the method of claim 1, Hall teaches the method of claim 1, wherein storing the position of the object in the three-dimensional local area in association with information identifying the object comprises: storing the position of the object in the three-dimensional local area (an example of a virtual model 600 describing local areas and parameters describing configuration of the local areas. The virtual model 600 may be stored in the virtual model database 505 of the mapping server 130. The virtual model 600 may represent geographic information storage area in the virtual model database 505 that stores geographically tied triplets of information (i.e., a local area identifier (ID) 605, a local area configuration ID 610, and a set of c parameters 615) for various local areas surrounding one or more headsets 110 - see col 18, lines 63-67 through col 19, lines 1-5) in association with one or more labels from detection of the object (for e.g. a unique local area 605, S1 – see col 19, lines 7-8 and Fig 6).
Regarding claim 11, Hall discloses a headset (see Fig 2) comprising: a frame (205 – see Fig 2); one or more display elements coupled to the frame (lens/electronic display 105 – see Fig 105), the one or more display elements configured to generate image light for presentation to a user (lens 105 may include an electronic display that display 2D or 3D images to a user – see col 4, lines 37-38); one or more imaging devices (150, passive camera assembly - see Fig 2) coupled to the frame, the one or more imaging devices configured to capture two-dimensional images of a three-dimensional local area surrounding the frame (note that the passive cameras can generate color ( e.g. , RGB ) image data – see col 7, lines 22-23); a depth camera assembly (140, depth camera assembly (DCA) – see Fig 2) configured to obtain depth information between the headset and portions of the three-dimensional local area (note that the DCA generate depth image data of a local area – see col 5, lines 27-29); and an object detection module (140, 180 and 146 – see Fig 1) including a processor (see col 17, lines 16) and a non-transitory computer readable storage medium (see col 24, line 8) having instructions encoded (see col 24, line 8) thereon that, when executed by the processor, cause the headset to:
detect an object in the local area from two-dimensional image of the local are captured by an imaging device of the one or more image devices(determine positions of an object – see col 9, lines 11-12);
determine a three-dimensional local area model of the local are from depth information (The DCA 140 generates depth image data of a local area, such as a room. Depth image data includes pixel values defining distance from the DCA 140,– see col 5, lines 28-30) generated by one or more depth sensors (140, depth camera – see Fig 2) included in the headset (see Fig 2), the three-dimensional local area model comprising a three-dimensional reconstruction of the three-dimensional local area to obtain a three-dimensional local area model of the local area (DCA 140 also provides a mapping of locations captured in the depth image data, such as a three-dimensional mapping of locations captured in the depth image data - see col 5, lines 30-33); and
store the position of the object in the three-dimensional local area (an example of a virtual model 600 describing local areas and parameters describing configuration of the local areas. The virtual model 600 may be stored in the virtual model database 505 of the mapping server 130. The virtual model 600 may represent geographic information storage area in the virtual model database 505 that stores geographically tied triplets of information (i.e., a local area identifier (ID) 605, a local area configuration ID 610, and a set of c parameters 615) for various local areas surrounding one or more headsets 110 - see col 18, lines 63-67 through col 19, lines 1-5) in association with information identifying the object (the virtual model 600 includes a listing of possible local areas S1, S2, . . . , Sn, each identified by a unique local area ID 605 – see col 19, lines 6-9); however, Hall does not explicitly teach determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
However, Khan explicitly teaches determining a bounding box for the object (see Fig. 1), the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object (estimated using 2D bounding box coordinate – see section 1, [p][003] and Fig. 1); and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device ([o]nce we obtain the 2D bounding box for the detected object in the image scene, we estimate the 3D cuboid for the object using the method – see section 3.2.3).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Khan of determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
Wherein having Hall`s determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
The motivation behind the modification would have been to create a virtual 3D space for enhancing human experience with reduction in power consumption hence portability since both Hall and Khan are methods for camera localization system and localization mapping of the real world. Wherein Hall selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Khan proposes an object localization solution in Three-Dimension (3D) for mobile devices using a 2D object detection Convolutional Neural Network (CNN) model with Augmented Reality (AR) technologies to recognize objects in the environment and determine their real-world coordinates (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Khan et al (NPL titled: Efficient and Scalable Object Localization in 3D on Mobile Device) Abstract).
Regarding claim 15, which corresponds to claims 5 except for reciting a different statutory category of a headset. Therefore, the rejection analysis of claim 5 are fully applicable to claim 15. Fig. 2 of Hall in combination illustrates a headset.
Regarding independent claim 19, Hall teaches a non-transitory computer-readable medium (see col 24, line 8) storing instructions (see col 24, line 8) that, when executed, cause:
capturing, at a headset worn (headset – see Fig 1) by a user (user - see col 3, lines 47), two-dimensional images of a local area surrounding the headset (an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” – see col 1, lines 61-64) by one or more imaging devices (150, passive camera assembly – see Fig 2) included in the headset (see Fig 2);
detecting an object in the local area from two-dimensional image of the local are captured by an imaging device of the one or more image devices (determine positions of an object – see col 9, lines 11-12);
determining a local area model of the local are from depth information (The DCA 140 generates depth image data of a local area, such as a room. Depth image data includes pixel values defining distance from the DCA 140,– see col 5, lines 28-30) generated by one or more depth sensors (140, depth camera – see Fig 2) included in the headset (see Fig 2), the local area model comprising a three-dimensional reconstruction of the local area to obtain a three-dimensional local area model of the local area (DCA 140 also provides a mapping of locations captured in the depth image data, such as a three-dimensional mapping of locations captured in the depth image data - see col 5, lines 30-33); and
storing the position of the object in the local area (an example of a virtual model 600 describing local areas and parameters describing configuration of the local areas. The virtual model 600 may be stored in the virtual model database 505 of the mapping server 130. The virtual model 600 may represent geographic information storage area in the virtual model database 505 that stores geographically tied triplets of information (i.e., a local area identifier (ID) 605, a local area configuration ID 610, and a set of c parameters 615) for various local areas surrounding one or more headsets 110 - see col 18, lines 63-67 through col 19, lines 1-5) in association with information identifying the object (the virtual model 600 includes a listing of possible local areas S1, S2, . . . , Sn, each identified by a unique local area ID 605 – see col 19, lines 6-9); however, Hall does not explicitly teach determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
However, Khan explicitly teaches determining a bounding box for the object (see Fig. 1), the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object (estimated using 2D bounding box coordinate – see section 1, [p][003] and Fig. 1); and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device ([o]nce we obtain the 2D bounding box for the detected object in the image scene, we estimate the 3D cuboid for the object using the method – see section 3.2.3).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Khan of determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
Wherein having Hall`s determining a bounding box for the object, the bounding box specifying dimensions of a region associated with the two-dimensional the image including the object; and determining a position of the object in the three-dimensional local area model based on the bounding box for the object and one or more parameters of the imaging device.
The motivation behind the modification would have been to create a virtual 3D space for enhancing human experience with reduction in power consumption hence portability since both Hall and Khan are methods for camera localization system and localization mapping of the real world. Wherein Hall selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Khan proposes an object localization solution in Three-Dimension (3D) for mobile devices using a 2D object detection Convolutional Neural Network (CNN) model with Augmented Reality (AR) technologies to recognize objects in the environment and determine their real-world coordinates (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Khan et al (NPL titled: Efficient and Scalable Object Localization in 3D on Mobile Device) Abstract).
Claims 2, 12 and 20 are rejected under 35 U.S.C. 103 as being anticipated by Hall et al (US Patent No.: 10972715) in view of in view of Khan et al (NPL titled: Efficient and Scalable Object Localization in 3D on Mobile Device) as applied to claims 1, 11 and 19 further in view of Gupta et al (US Patent No.: 11417069) further in view of Majercik et al (NPL titled: A Ray-Box Intersection Algorithm and Efficient Dynamic Voxel Rendering).
Regarding to claim 2, Hall in view of Khan teaches the method of claim 1, Hall in view of Khan fail to explicitly teach wherein determining the position of the object in the local area model based on the bounding box for the object and one or more parameters of the imaging device comprises: determining a center of the bounding box based on dimensions of the bounding box; generating a ray intersecting the center of the bounding box in coordinates of the local area model based on parameters of the imaging device; and determining the position of the object in the local area model as a position in the local area model that the ray intersects.
However, Gupta explicitly teaches an object and camera localization method including wherein determining the position of the object in the local area model based on the bounding box for the object and one or more parameters of the imaging device comprises: determining a center of the bounding box based on dimensions of the bounding box (using the cuboid 206, the camera device 104 can generate or calculate the centroid 208 of the cuboid 206. The centroid 208 is the center of the cuboid 206 in 3D space – see col 16, lines 50-53); generating a ray of the bounding box in coordinates of the local area model based on parameters of the imaging device (the camera height is calculated (generated) by pointing the camera device 104 (e.g., shooting a ray from the center of the screen or the camera 522) towards a detected plane on the floor – see col 18, lines 19-22); and determining the position of the object in the local area model as a position in the local area model (generate a 3D map which includes the virtual object 202 in the exact same (within suitable thresholds) virtual location as the real object 202 in the real world – see col 15, lines 45-49 and see Fig 10C).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall as modified by Khan of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Gupta wherein determining the position of the object in the local area model based on the bounding box for the object and one or more parameters of the imaging device comprises: determining a center of the bounding box based on dimensions of the bounding box; generating a ray intersecting the center of the bounding box in coordinates of the local area model based on parameters of the imaging device; and determining the position of the object in the local area model as a position in the local area model that the ray intersects..
Wherein having Hall the object and camera localization method includes determining a bounding box for the object, the bounding box specifying dimensions of a region of the image including the object; and determining a position of the object in the local area model based on the bounding box for the object and one or more parameters of the imaging device.
The motivation behind the modification would have been to create a virtual 3D space for enhancing human experience with reduction in power consumption hence portability since both Hall and Gupta are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Gupta’s localization method is used for generating an object label of the object and a bounding box of the object in the image by generating anchor points in real world coordinates of the real 3D space of the object thus creating a virtual 3D map.
(Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Gupta et al (Pub No.: US20210390756) Abstract).
Hall in view of Gupta fails to explicitly teach wherein the generating a ray that intersecting the center and determining the position include the ray intersects.
Majercik explicitly teaches a method for rendering large models wherein the generating a ray that intersecting the center and determining the position include the ray intersects (ray-box intersection algorithm and box center – see section 5).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall in view of Gupta of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Majercik wherein the generating a ray that intersecting the center and determining the position include the ray intersects.
Wherein having Hall, the object and camera localization method includes wherein the generating a ray that intersecting the center and determining the position include the ray intersects.
The motivation behind the modification would have been for efficiently rendering large model while enhancing the human viewing experience with reduction in power consumption hence portability since both Hall and Gupta are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Majercik rendering method is used for generating a novel and efficient method for rendering large models composed of individually-oriented voxels (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Majercik et al (NPL titled: A Ray-Box Intersection Algorithm and Efficient Dynamic Voxel Rendering) Abstract).
Regarding claim 12, which corresponds to claims 2 except for reciting a different statutory category of a headset. Therefore, the rejection analysis of claim 2 are fully applicable to claim 12. Fig. 2 of Hall in combination illustrates a headset.
Regarding claim 20, which corresponds to claims 2 except for reciting a different statutory category of a headset. Therefore, the rejection analysis of claim 2 is fully applicable to claim 20.
Claims 6, 8-10, 16 and 18 are rejected under 35 U.S.C. 103 as being anticipated by Hall et al (US Patent No.: 10972715) in view of Khan et al (NPL titled: Efficient and Scalable Object Localization in 3D on Mobile Device) in view of Gupta et al (Pub No.: 11417069) as applied to claims 1, and 11 and 19 further in view of Border et al (Pub No.: 20130278631).
Regarding to claim 6, Hall in view of Khan teaches the method of claim 1, Hall in view of Khan fail to explicitly teach, further comprising: receiving an input at the headset from the user identifying the object and requesting navigation to the object; and displaying at least a portion of the generated directions to the user via one or more display elements.
However, Gupta explicitly teaches an object and camera localization method including further comprising: receiving an input at the headset from the user identifying the object and requesting navigation to the object (the user can use the input device to virtually navigate to the virtual location of the virtual object. The virtual location has virtual world coordinates that correspond to real world coordinates in the world (e.g. planet Earth) or the user can physically move to the same real location as the real object – see col 24, lines 17-19 and lines 33-34);; and displaying at least a portion of the generated directions to the user via one or more display elements (the cave projection can also display coordinates of the particular location of the POV being displayed, for example using real world coordinates of latitude and longitude (and optionally height) – see col 24, lines 50-53).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall as modified by Khan of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Gupta receiving an input at the headset from the user identifying the object and requesting navigation to the object; and displaying at least a portion of the generated directions to the user via one or more display elements.
Wherein having Hall the object and camera localization method includes receiving an input at the headset from the user identifying the object and requesting navigation to the object; and displaying at least a portion of the generated directions to the user via one or more display elements.
The motivation behind the modification would have been to create a virtual 3D space for enhancing human experience with reduction in power consumption hence portability since both Hall and Gupta are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Gupta’s localization method is used for generating an object label of the object and a bounding box of the object in the image by generating anchor points in real world coordinates of the real 3D space of the object thus creating a virtual 3D map.
(Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Gupta et al (Pub No.: US20210390756) Abstract).
Hall in view of Khan and Gupta fail to explicitly teach retrieving the stored position of the object in the local area model; generating directions from the current location of the headset in the local area model to the stored position of the object in the local area model; and determining a current location of the headset in the local area model from the depth information.
However, Border explicitly teaches retrieving the stored position of the object in the local area model (The video output may thus be used to help determine the user's location, or may be sent remotely to others to assist in helping to locate the location of the wearer, or for any other purpose – see [p][0502]); generating directions from the current location of the headset in the local area model to the stored position of the object in the local area mode (the wearer may need assistance in navigating to a location, whether in a car, on a bike, walking, or the like – see [p][0469]); and determining a current location of the headset in the local area model from the depth information (control signal could be generated by a location specified by the wearer, by what is currently being displayed in the glasses or on a destination spoken by wearer – see [p][0469]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall in view of Khan and Gupta of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Border wherein retrieving the stored position of the object in the local area model; generating directions from the current location of the headset in the local area model to the stored position of the object in the local area model; and determining a current location of the headset in the local area model from the depth information.
Wherein having Hall, retrieving the stored position of the object in the local area model; generating directions from the current location of the headset in the local area model to the stored position of the object in the local area model; and determining a current location of the headset in the local area model from the depth information.
The motivation behind the modification would have been for efficiently determining a physical location while enhancing the human viewing experience with reduction in power consumption hence portability since both Hall and Border are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Border determines a physical location of the user and a head mounted display device, and identifying and determining a distance from the user to one or more objects of interest in the user's field of view (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Border et al (Pub No.: 20130278631), Abstract).
Regarding claim 8, Hall in view of Khan and Gupta teach the method of claim 1, Hall in view of Khan and Gupta fails to explicitly further comprising: displaying an interface element to the user via one or more display elements of the headset, the interface element displayed in a position in the local area model relative to the position of the object in the local area model.
However, Border explicitly teaches displaying an interface element to the user via one or more display elements of the headset, the interface element displayed in a position in the local area model relative to the position of the object in the local area model (see Fig 23).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall in view of Khan and Gupta of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Border of displaying an interface element to the user via one or more display elements of the headset, the interface element displayed in a position in the local area model relative to the position of the object in the local area model.
Wherein having Hall, displaying an interface element to the user via one or more display elements of the headset, the interface element displayed in a position in the local area model relative to the position of the object in the local area model.
The motivation behind the modification would have been for efficiently determining a physical location while enhancing the human viewing experience with reduction in power consumption hence portability since both Hall and Border are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Border determines a physical location of the user and a head mounted display device, and identifying and determining a distance from the user to one or more objects of interest in the user's field of view (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Border et al (Pub No.: 20130278631), Abstract).
Regarding claim 9, Hall in view of Khan and Gupta teach the method of claim 1, Hall in view of Khan and Gupta fails to explicitly wherein the position in the local area model where the interface element is displayed is determined as an offset from a portion of the bounding box for the object.
However, Border explicitly teaches wherein the position in the local area model where the interface element is displayed is determined as an offset from a portion of the bounding box for the object (see Fig 23 and [p][0474]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall in view of Khan and Gupta of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Border wherein the position in the local area model where the interface element is displayed is determined as an offset from a portion of the bounding box for the object.
Wherein having Hall, wherein the position in the local area model where the interface element is displayed is determined as an offset from a portion of the bounding box for the object.
The motivation behind the modification would have been for efficiently determining a physical location while enhancing the human viewing experience with reduction in power consumption hence portability since both Hall and Border are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Border determines a physical location of the user and a head mounted display device, and identifying and determining a distance from the user to one or more objects of interest in the user's field of view (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Border et al (Pub No.: 20130278631), Abstract).
Regarding claim 10, Hall in view of Khan and Gupta teach the method of claim 8, Hall in view of Khan and Gupta fails to explicitly, wherein a portion of the interface element contacts a portion of the bounding box for the object in the local area model.
However, Border explicitly teaches wherein a portion of the interface element contacts a portion of the bounding box for the object in the local area model (the user may be able to change the view perspective of the 3D displayed image 1512C, such as by turning their head, and where the live external environment and the 3D displayed image stay together even as the user turns their head, moves their position, and the like. In this way, the eyepiece may be able to provide an augmented reality by overlaying information onto the user's viewed external environment, such as the overlaid 3D displayed map 1512C, the location information 1514C, and the like, where the displayed map, information, and the like, may change as the user's view change – see [p][0642] and Fig 15 C).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Hall in view of Khan and Gupta of having the object and camera localization method for using a controller identifies a region of interest within an image captured by an imaging device of a local area and apply one or more stereo imaging processes, also referred to as “stereo processes,” to a subset of pixels corresponding to the region of interest with the teachings of Border wherein a portion of the interface element contacts a portion of the bounding box for the object in the local area model.
Wherein having Hall, wherein a portion of the interface element contacts a portion of the bounding box for the object in the local area model.
The motivation behind the modification would have been for efficiently determining a physical location while enhancing the human viewing experience with reduction in power consumption hence portability since both Hall and Border are methods for camera localization system and localization mapping of the real world. Wherein Hall’s selective retrieval or processing of data from a subset of pixels allows the controller to reduce power consumption by the DCA, as well as reduce bandwidth for communication between the imaging sensors and the controller, without impairing determination of depth information for the local area by the DCA, while Border determines a physical location of the user and a head mounted display device, and identifying and determining a distance from the user to one or more objects of interest in the user's field of view (Please see Hall et al (US Patent No.: 10972715), col 2, lines 13-18 and Border et al (Pub No.: 20130278631), Abstract).
Regarding claim 16, which corresponds to claims 6 except for reciting a different statutory category of a headset. Therefore, the rejection analysis of claim 6 are fully applicable to claim 16. Fig. 2 of Hall in combination illustrates a headset.
Regarding claim 18, which corresponds to claims 8 except for reciting a different statutory category of a headset. Therefore, the rejection analysis of claim 8 are fully applicable to claim 18. Fig. 2 of Hall in combination illustrates a headset.
Allowable Subject Matter
Claims 3-4, 7, 13-14 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gupta et al (US Patent No.: 11417069 ) discloses object and camera localization system and localization method for mapping of the real world. The mapping can be done in real-time or near real-time to the detection of the real objects by a camera device which is used to capture one or more images of an object. The localization method can be used to generate an object label of the object and a bounding box of the object in the image. The localization method can be used to generate anchor points in real world coordinates of the real 3D space of the object, a cuboid of the object, and a centroid of the cuboid. A virtual 3D map can be generated that which includes the location and pose of the real object in the real-world coordinates.
Gupta et al (US Patent No.: 11776206) discloses an extended reality system and extended reality method for digital twins, in which the digital twin can be a virtual asset of a real asset. The real asset can be a real object. For example, an initiation of an event in relation to the real asset causes the extended reality method to generate one or more predicted virtual states which are predicted to achieve the event in the virtual asset. The event can be initiated through the real asset and through the virtual asset. The extended reality method can receive one or more further real states of the real asset which achieve the event. The extended reality method can generate a reality 3D map in an extended reality application which concurrently displays, in the 3D space, the virtual asset in the one or more predicted virtual states and the real asset in the one or more further real states.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDRAE S ALLISON whose telephone number is (571)270-1052. The examiner can normally be reached on Monday-Friday 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns, can be reached on (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDRAE S ALLISON/Primary Examiner, Art Unit 2673
May 28, 2026
Prosecution Insights