Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required: claimed feature “cull one or more of the voxels to identify ones of the voxels to include in a volumetric scene to be rendered” must be described in the specification.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 2-21 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. The originally filed disclosure does not appear to recite or suggest to "cull one or more of the voxels to identify ones of the voxels to include in a volumetric scene to be rendered". The closest supporting paragraph appears to be paragraph 38, which sets forth "empty space in the volumetric representation, which may be included in one or more of the voxels, can be culled from the volumetric representation", which is different than the claimed limitation.
Thus, one of ordinary skill in the art before the effective filing date of the claimed invention would not know how to make and/or use the claimed invention without undue experimentation.
The rejected language will be interpreted as voxels being culled, removing unwanted voxels to be rendered, which would result in remaining voxels being the ones to be displayed, therefore being identified.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Particularly, the phrase "cull one or more of the voxels to identify ones of the voxels to include in a volumetric scene to be rendered" is unclear. The closest supporting paragraph appears to be paragraph 38, which sets forth "empty space in the volumetric representation, which may be included in one or more of the voxels, can be culled from the volumetric representation", which is different than the claimed limitation. It is unclear as to how the process of culling voxels leads to an identification of voxels to be rendered in a volumetric scene.
The dependent claims to not cure the noted deficiency, and are therefore rejected with similar rationale to the independent claims which contain this rejected language.
The rejected language will be interpreted as voxels being culled, removing unwanted voxels to be rendered, which would result in remaining voxels being the ones to be displayed, therefore being identified.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 2-3, 9-10, and 16-17 are rejected under 35 U.S.C. 102a as being anticipated by Petrovskaya et al. (U.S. Pg Pub No. US 20160148433 A1), hereinafter referred to as Petrovskaya.
Regarding Claim 2, Petrovskaya teaches An apparatus comprising: a headset including at least one video display; (Petrovskaya [0298] “Users may observe virtual objects using a viewing device, e.g., a smart mobile device (a phone or tablet) or a head-mounted display (HMD), such as helmet or goggles (e.g., GoogleGlass®).”) machine-readable instructions; (Petrovskaya [0436] “The memory 6310 and storage devices 6320 are computer-readable storage media that may store instructions that implement at least portions of the various embodiments.”) and at least one processor circuit to be programmed based on the machine-readable instructions to: (Petrovskaya [0438] “The various embodiments introduced herein can be implemented by, for example, programmable circuitry (e.g., one or more microprocessors) programmed with software and/or firmware, or entirely in special-purpose hardwired (non-programmable) circuitry, or in a combination of such forms.”) access a data structure representative of voxels of a three- dimensional volume; (Petrovskaya [0073] “An application running on the capture device 105a or on a remote system 125 in communication with the capture device 105a via a network 120 may integrate 160 the frames in the data log to form a three-dimensional internal model representation 130 (e.g., one or more vertex meshes represented here in a top-down view 100b). This integration, also referred to as “mapping” herein, may be performed on the capture device 105a or on the remote device 125 or on a combination of the two.” [0084] “At block 415, the mapping system may also apply any desired post-processing operations, e.g., map coloring. Post processing may also involve the creation of data structures facilitating tracking as discussed in greater detail herein.” Data structure can be accessed, see paragraphs [119], [128], [187]) cull one or more of the voxels to identify ones of the voxels to include in a volumetric scene to be rendered; (Petrovskaya [0108] “In some embodiments, any points/pixels contained in a “border” area (around the edge of the captured depth image, where the edge could be of some pixel width, e.g., constant, or some distance after skipping any part of the edge where there are no pixels containing depth data, etc.) may be filtered out, or removed from consideration, and hence not considered by the correspondence identification 850a process. This would reduce the amount of previously unseen “new data” appearing in a depth frame relative to a previously acquired and processed depth frames.” [0364] “Some embodiment may back-cull the polygons of the 3D model to not render polygons facing away from the user, to allow a user to look "into the model" from outside.” Examiner Note: This element is being interpreted as voxels being culled, removing unwanted voxels to be rendered, which would result in remaining voxels being the ones to be displayed, therefore being identified. The polygons(voxels) being culled here that allow the user to look into the model is a form of removing the voxels to leave behind the intended visible area.) access data for the ones of the voxels to include in the volumetric scene; (Petrovskaya [0085] “For example, a development toolkit may be provided to the AR developer, allowing them to access the 3D representation.” [0364] “Some embodiment may back-cull the polygons of the 3D model to not render polygons facing away from the user, to allow a user to look "into the model" from outside.” Examiner Note: Allowing the user to look into the model requires the voxels to be included in the volumetric scene to be accessed.) render, based on the data, at least one of an augmented reality scene or a mixed reality scene; (Petrovskaya [0074] “An augmented reality (AR) device 105b (which may be the same as the capture device 105b) may then use 170 the model 130 in conjunction with incoming depth frame data to present an augmented reality experience 100c.”) and output the at least one of the augmented reality scene or the mixed reality scene to the at least one video display of the headset. (Petrovskaya [0074] “As real-time images are captured of the environment 150 and displayed on the AR device…” [0298] “Users may observe virtual objects using a viewing device, e.g., a smart mobile device (a phone or tablet) or a head-mounted display (HMD), such as helmet or goggles (e.g., GoogleGlass®).”)
Regarding Claim 3, Petrovskaya teaches the apparatus of claim 2, wherein the data includes truncated signed distance function (TSDF) data, (Petrovskaya [0119] “The intermediate representation may be, e.g., a truncated signed distance function (TSDF) data structure (though one will readily recognize other suitable data structures).” See also paragraphs [0120] [0121] [0122) and one or more of the at least one processor circuit is to render the at least one of the augmented reality scene or the mixed reality scene based on the TSDF data. (Petrovskaya [0122] “Once all, or a sufficient number (e.g., to complete a model), of frames have been considered at block 1020, at block 1035 the system may build a 3D mesh out of the TSDF representation” [0149] “In this case, for each pose of the camera, the system may generate a virtual depth scan using a 3D model of the mesh, e.g., an OpenGL™ rendering of the 3D mesh model into the depth buffer and then reading back the depth values.” [0074] “An augmented reality (AR) device 105b (which may be the same as the capture device 105b) may then use 170 the model 130 in conjunction with incoming depth frame data to present an augmented reality experience 100c.”)
Claim 9 is an article of manufacture claim (Petrovskaya [0436] teaches machine-readable instructions being stored in a computer-readable storage media) substantially equivalent to apparatus claim 2, and is therefore rejected under the same rationale.
Claim 10 recites similar limitations to Claim 3, and is therefore rejected under similar rationale.
Claim 16 is an apparatus claim (Petrovskaya [0438] “The computing system 6300 may include one or more central processing units (“processors”) 6305, memory 6310, input/output devices 6325 (e.g., keyboard and/or pointing devices and/or touchscreen devices, display devices, etc.), storage devices 6320 (e.g., disk drives), and network adapters 6330 (e.g., network interfaces) that are connected to an interconnect 6315.”) substantially equivalent to apparatus Claim 2, and is therefore rejected under the same rationale.
Claim 17 recites similar limitations to Claim 3, and is therefore rejected under similar rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-8, 11-15, and 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over Petrovskaya et al. (U.S. Pg Pub No. US 20160148433 A1), hereinafter referred to as Petrovskaya, in view of Laine et al. (“Efficient sparse voxel octrees”), hereinafter referred to as Laine.
Regarding Claim 4, Petrovskaya fails to teach the apparatus of claim 2, wherein one or more of the at least one processor circuit is to: make one or more occupancy decisions associated with the data structure; and cull the one or more of the voxels based on the one or more occupancy decisions.
Laine teaches make one or more occupancy decisions associated with the data structure; (Laine Page 56, Section 3.1 “We encode the topology of the octree using 64-bit child descriptors, each corresponding to a single non-leaf voxel. Leaf voxels do not require a descriptor of their own, as they are described by their parents. As illustrated in Figure 2, the child descriptors are divided into two 32-bit parts. The first part describes the set of child voxels, while the second part is related to contours (Section 3.2). Each voxel is subdivided spatially into 8 child slots of equal size. The child descriptor contains two bitmasks, each storing one bit per child slot. valid mask tells whether each of the child slots actually contains a voxel, while leaf mask further specifies whether each of these voxels is a leaf. Based on the bitmasks, the status of a child slot can be interpreted as follows: Neither bit is set: the slot is not intersected by a surface. The bit in valid mask is set: the slot contains a non-leaf voxel. Both bits are set: the slot contains a leaf voxel.” Examiner Note: Checking if a slot is intersected or not is checking for occupancy information of that voxel.)
and cull the one or more of the voxels based on the one or more occupancy decisions. (Laine Page 59 Section 4.1 “Line 9 computes the span tc corresponding to the current cube to be used by INTERSECT and ADVANCE, and line 10 checks whether to process the current voxel or skip it. If the bit corresponding to the voxel in valid mask is not set, or the active span t is empty, the code determines that the ray cannot intersect the voxel and skips directly to ADVANCE.”)
Petrovskaya and Laine are analogous in the art of voxel processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Petrovskaya’s system of culling voxels with Laine’s data structure that contains occupancy information with bitmaps to decide whether to cull or not, to minimize the memory footprint while supporting efficient ray casts (Laine Page 56 Section 3).
Regarding Claim 5, Petrovskaya fails to teach the apparatus of claim 2, wherein the data is first data, and the data structure includes second data to indicate whether the voxels are occupied.
Laine teaches wherein the data is first data, and the data structure includes second data to indicate whether the voxels are occupied. (Laine Page 56, Section 3.1 “We encode the topology of the octree using 64-bit child descriptors, each corresponding to a single non-leaf voxel. Leaf voxels do not require a descriptor of their own, as they are described by their parents. As illustrated in Figure 2, the child descriptors are divided into two 32-bit parts. The first part describes the set of child voxels, while the second part is related to contours (Section 3.2). Each voxel is subdivided spatially into 8 child slots of equal size. The child descriptor contains two bitmasks, each storing one bit per child slot. valid mask tells whether each of the child slots actually contains a voxel, while leaf mask further specifies whether each of these voxels is a leaf. Based on the bitmasks, the status of a child slot can be interpreted as follows: Neither bit is set: the slot is not intersected by a surface. The bit in valid mask is set: the slot contains a non-leaf voxel. Both bits are set: the slot contains a leaf voxel.” Examiner Note: Checking if a slot is intersected or not is checking for occupancy information of that voxel.)
Petrovskaya and Laine are analogous in the art of voxel processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Petrovskaya’s system of culling voxels with Laine’s data structure that contains occupancy information with bitmaps to decide whether to cull or not, to minimize the memory footprint while supporting efficient ray casts (Laine Page 56 Section 3).
Regarding Claim 6, the apparatus of claim 5, wherein Laine further teaches the second data includes bits to indicate whether associated ones of the voxels are occupied. (Laine Page 56, Section 3.1 “We encode the topology of the octree using 64-bit child descriptors, each corresponding to a single non-leaf voxel. Leaf voxels do not require a descriptor of their own, as they are described by their parents. As illustrated in Figure 2, the child descriptors are divided into two 32-bit parts. The first part describes the set of child voxels, while the second part is related to contours (Section 3.2). Each voxel is subdivided spatially into 8 child slots of equal size. The child descriptor contains two bitmasks, each storing one bit per child slot. valid mask tells whether each of the child slots actually contains a voxel, while leaf mask further specifies whether each of these voxels is a leaf. Based on the bitmasks, the status of a child slot can be interpreted as follows: Neither bit is set: the slot is not intersected by a surface. The bit in valid mask is set: the slot contains a non-leaf voxel. Both bits are set: the slot contains a leaf voxel.” Examiner Note: Checking if a slot is intersected or not is checking for occupancy information of that voxel.)
Petrovskaya and Laine are analogous in the art of voxel processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Petrovskaya’s system of culling voxels with Laine’s data structure that contains occupancy information with bitmaps to decide whether to cull or not, to minimize the memory footprint while supporting efficient ray casts (Laine Page 56 Section 3).
Regarding Claim 7, Petrovskaya teaches the apparatus of claim 6, wherein one or more of the at least one processor circuit is to determine whether to perform or skip an operation associated with a first one of the voxels Petrovskaya [0161] At block 1540, the system may compare the range to vertex with the measured range, e.g., to determine if the depth taken with the RGB image suggests that the vertex under consideration is occluded or otherwise not visible. For example, the vertex under consideration may fall behind another object (e.g., be occluded) relative to the pose at which the RGB image was taken (e.g., the range may be determined as a threshold of the absolute difference). If so, the vertex color should not be updated. Similarly, the sensor data may be bad, and the vertex may be "precluded" in that the depth value is far past the vertex's position, suggesting that the data is bad and should be ignored.)
However, Petrovskaya fails to teach based on at least one value of at least one of the bits.
Laine teaches wherein one or more of the at least one processor circuit is to determine whether to perform or skip an operation associated with a first one of the voxels based on at least one value of at least one of the bits. (Laine Page 59 Section 4.1 “Line 9 computes the span tc corresponding to the current cube to be used by INTERSECT and ADVANCE, and line 10 checks whether to process the current voxel or skip it. If the bit corresponding to the voxel in valid mask is not set, or the active span t is empty, the code determines that the ray cannot intersect the voxel and skips directly to ADVANCE.” Examiner Note: It is being determined whether to perform or skip the processing(operation) associated with a voxel.)
Petrovskaya and Laine are analogous in the art of voxel processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Petrovskaya’s system of culling voxels with Laine’s data structure that contains occupancy information with bitmaps to decide whether to skip operations corresponding to a voxel, as storing information with bits is a common method of storing data or making decisions; and to minimize the memory footprint while supporting efficient ray casts (Laine Page 56 Section 3).
Regarding Claim 8, Petrovskaya teaches the apparatus of claim 6, wherein one or more of the at least one processor circuit is to determine whether to cull a first one of the voxels Petrovskaya [0108] “In some embodiments, any points/pixels contained in a “border” area (around the edge of the captured depth image, where the edge could be of some pixel width, e.g., constant, or some distance after skipping any part of the edge where there are no pixels containing depth data, etc.) may be filtered out, or removed from consideration, and hence not considered by the correspondence identification 850a process.”)
However, Petrovskaya fails to teach based on at least one value of at least one of the bits.
Laine teaches wherein one or more of the at least one processor circuit is to determine whether to cull a first one of the voxels based on at least one value of at least one of the bits. (Laine Page 59 Section 4.1 “Line 9 computes the span tc corresponding to the current cube to be used by INTERSECT and ADVANCE, and line 10 checks whether to process the current voxel or skip it. If the bit corresponding to the voxel in valid mask is not set, or the active span t is empty, the code determines that the ray cannot intersect the voxel and skips directly to ADVANCE.” Examiner Note: It is being determined whether a voxel would be skipped, if a voxel is skipped it is functionally culled.)
Petrovskaya and Laine are analogous in the art of voxel processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Petrovskaya’s system of culling voxels with Laine’s data structure that contains occupancy information with bitmaps to decide whether to cull or not, as storing information with bits is a common method of storing data or making decisions; and to minimize the memory footprint while supporting efficient ray casts (Laine Page 56 Section 3).
Claim 11 recites similar limitations to Claim 4, and is therefore rejected under similar rationale.
Claim 12 recites similar limitations to Claim 5, and is therefore rejected under similar rationale.
Regarding Claim 13, the article of manufacture of claim 12, wherein Laine further teaches the second data includes a first bit and a second bit, the first bit to indicate whether a first one of the voxels is occupied, the second bit to indicate whether a second one of the voxels is occupied. (Laine Page 56, Section 3.1 “The first part describes the set of child voxels, while the second part is related to contours (Section 3.2). Each voxel is subdivided spatially into 8 child slots of equal size. The child descriptor contains two bitmasks, each storing one bit per child slot. valid mask tells whether each of the child slots actually contains a voxel, while leaf mask further specifies whether each of these voxels is a leaf. Based on the bitmasks, the status of a child slot can be interpreted as follows: Neither bit is set: the slot is not intersected by a surface. The bit in valid mask is set: the slot contains a non-leaf voxel. Both bits are set: the slot contains a leaf voxel.” Examiner Note: The “set of child voxels” indicates multiple voxels, and the bits corresponding are set to indicate occupancy.)
Petrovskaya and Laine are analogous in the art of voxel processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have combined Petrovskaya’s system of culling voxels with Laine’s data structure that contains occupancy information with bitmaps to decide whether to cull or not, to minimize the memory footprint while supporting efficient ray casts (Laine Page 56 Section 3).
Claim 14 recites similar limitations to Claim 7, and is therefore rejected under similar rationale.
Claim 15 recites similar limitations to Claim 8, and is therefore rejected under similar rationale.
Claim 18 recites similar limitations to Claim 4, and is therefore rejected under similar rationale.
Claim 19 recites similar limitations to Claim 5, and is therefore rejected under similar rationale.
Claim 20 recites similar limitations to Claim 13, and is therefore rejected under similar rationale.
Claim 21 recites similar limitations to Claim 7, and is therefore rejected under similar rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID W SOON whose telephone number is (571)272-8113. The examiner can normally be reached M-F 7:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID W SOON/ Examiner, Art Unit 2615
/ALICIA M HARRINGTON/ Supervisory Patent Examiner, Art Unit 2615