Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Art Unit – Location
The Art Unit location of your application in the USPTO may have changed. To aid in correlating any papers for this application, all further correspondence regarding this application should be directed to Art Unit 2682.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because:
The claimed invention is directed to an abstract idea without significantly more.
The claim(s) recite(s) mathematical concepts, a mental process, or certain methods of organizing human activity. This judicial exception is not integrated into a practical application because the abstract idea is implemented using generic machine learning.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claimed methods can be performed by a human using conventional tools.
Step 1. The claims are directed to a Process, Machine (device), and Article of Manufacture.
Step 2A. Prong 1. The invention comprises an Abstract Idea directed to: Mathematical Concepts, a Mental Process, and/or Certain Methods of Organizing Human Activity.
Mathematical Concepts of: “A depth map including a … plurality of pixels”, “smoothing a depth map based on a world-fixed vector” and “transforming.. the image” are mathematical concepts.
Please refer to MPEP 2106.04(a)(2) I C. e.g. “Mathematical Calculations”.
A Mental Process of: “capturing an image.. of a physical environment”, “obtaining a depth map … of the image”, “smoothing the depth map”, “transforming … the image” can be performed in the mind of a human. MPEP 2106.04(a)(2) III B. e.g. “A Claim That Encompasses a Human Performing the Step(s) Mentally With or Without a Physical Aid Recites a Mental Process”.
A Method of Organizing Human Activity of “capturing an image”, performing the mental and mathematical processes above with or without physical aids and “displaying the transformed image” are following procedural steps. E.g. “following rules or instructions” MPEP 2106.04(a)(2) II C.
Step 2A Prong 2. There are no additional elements which are directed to integration into a practical application, a technical improvement, or a technical environment. 2106.04(d)(1), 2106.05.
Step 2B. Are there additional elements that amount to significantly more? No.
The claims cite a process which can be performed by a human using mathematical concepts, mental processes, and following known rules or instructions of human activity; where the inclusion of devices such as an image sensor, display, and a processor for filtering is generic and lacks the details for a significant technical improvement or an inventive concept. MPEP 2106.05. Dependent claim limitations of a vector direction, image analysis, and filtering are mathematical concepts which can be performed in the human mind with or without conventional tools.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al (US 2019/0101758 A1) “Zhu” in view of Eble et al. (US 10,832,427 B1) “Eble”.
1. Zhu teaches: A method ("A method for reconstructing a perspective captured in an image" [Claim 16]) comprising: at a device (a “HMD” Head Mounted Display “100A” [0041] Fig. 1) including an image sensor (including “one or more sensors” [0042]), a display (“HMD” [0041]) , one or more processors ("at least one hardware processing unit 105" [0042]; “Processor(s) 105” Fig. 1.) , and non-transitory memory ("storage 125” [0042] Fig. 1.) : capturing, using the image sensor, an image of a physical environment ("visualizations 320 of the real-world objects 310 are being rendered on the user's HMD" [0059]) ; obtaining a depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment ("depth map is comprised of a plurality of three-dimensional coordinates 1005, where each three-dimensional coordinate corresponds to a single pixel included within a plurality of pixels that together form a particular image." [0089]) ; smoothing the depth map (" “smooth” depth map by performing the temporal smoothing actions of filtering" [0093].)
Zhu does not explicitly teach where the depth map is based on a world-fixed vector; transforming, using the one or more processors, the image of the physical environment based on the smoothed depth map;
However, Eble teaches the depth map is based on a world-fixed vector; transforming, using the one or more processors, the image of the physical environment based on the smoothed depth map ("transforming the image of the scene includes determining, for a pixel having a pixel location, a new pixel location by multiplying a vector including the pixel location and the depth by a transformation matrix representing a difference between the point-of-view of the scene camera and the point-of-view of the user." [Col. 11 lines 37-42])
The depth map of Zhu can be modified by Eble to be based on a world-fixed vector as the point-of-view of a user.
The motivation for the combination is provided by Eble to present a real world image to a user which is substantially similar to what the user would see to avoid impaired distance perception, disorientation, and poor hand-eye coordination due to different positions of the users eyes. [Col. 1 lines 57-62].
Furthermore, the combination of Zhu and Eble teach: and displaying, on the display, the transformed image (“visualization is then displayed on a HMD” [0033] of Zhu; “displaying, on the display, the transformed image" [ABSTRACT of Eble.]) .
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
2. The method of claim 1, wherein the world-fixed vector is a gravity vector of the physical environment (The world-fixed vector is a gravity vector shown by a person standing a downward gravity direction [Figure 5] of Eble.) .
The depth map of Zhu can be modified by Eble to be based on a world-fixed vector as the point-of-view of a user.
The motivation for the combination is provided by Eble to present a real world image to a user which is substantially similar to what the user would see to avoid impaired distance perception, disorientation, and poor hand-eye coordination due to different positions of the users eyes. [Col. 1 lines 57-62].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
3. The method of claim 1, wherein the world-fixed vector is a vector parallel or perpendicular to an object in the physical environment (The world-fixed vector is a vector parallel or perpendicular to the walls or floor of the room in [Figure 5] of Eble.) .
The depth map of Zhu can be modified by Eble to be based on a world-fixed vector as the point-of-view of an object in a physical environment.
The motivation for the combination is provided by Eble to present a real world image to a user which is substantially similar to what the user would see to avoid impaired distance perception, disorientation, and poor hand-eye coordination due to different positions of the users eyes. [Col. 1 lines 57-62].
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
Claims 4, 5, 6, 10, 11, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al (US 2019/0101758 A1) “Zhu” in view of Eble et al. (US 10,832,427 B1) “Eble” and further in view of Tam et al. (US 2013/0009952 A1) “Tam”.
4. Zhu and Eble teach: The method of claim 1, further comprising determining a smoothing direction for the image of the physical environment corresponding to the world-fixed vector as in claims 2 and 3.
Zhu and Eble do not explicitly teach: wherein the depth map is maximally smoothed in the smoothing direction more than in a direction perpendicular to the smoothing direction.
However, Tam teaches: wherein the depth map is maximally smoothed in the smoothing direction more than in a direction perpendicular to the smoothing direction ("Table I gives exemplary filter parameter values which are three times, larger in the vertical than the horizontal orientation." [0110])
The smoothing of Zhu and Eble can be modified to asymmetrically smooth an image.
The motivation for the combination is provided by Tam “ the fact that the human visual system is more attuned to horizontal disparities than vertical disparities”.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
5. The method of claim 4, wherein determining the smoothing direction includes determining the world-fixed vector (the smoothing direction can be vertical or horizontal in Tam based on the world-fixed vector of Eble as in claims 2 and 3.) .
The smoothing of Zhu and Eble can be modified to align with a world-fixed vector.
The motivation for the combination is provided by Tam “ the fact that the human visual system is more attuned to horizontal disparities than vertical disparities” due to alignment with gravity.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
6. The method of claim 5, wherein determining the world-fixed vector includes determining the world-fixed vector using an inertial measurement unit ("variation in the position and the movement of the objects, such as a detected change in position, velocity, orientation, or acceleration" [0048] of Zhu.) .
The smoothing of Zhu and Eble can be modified to align with an inertial measurement unit.
The motivation for the combination is provided by Tam “ the fact that the human visual system is more attuned to horizontal disparities than vertical disparities” due to alignment with an inertial frame of reference.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
10. The method of claim 4, wherein the smoothing direction forms an angle with a vertical vector in an image space of the image of the physical environment ("vertical” and “horizontal” orientation shown in Table 1 of Tam [0110]).
The smoothing of Zhu and Eble can be modified to smooth an image at an angle (e.g. horizontal) with the vertical vector in an image of the physical environment.
The motivation for the combination is provided by Tam “ the fact that the human visual system is more attuned to horizontal disparities than vertical disparities”.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
11. The method of claim 10, wherein the angle is non-zero ("vertical” and “horizontal” orientation shown in Table 1 of Tam [0110] where vertical and horizontal is 90 degrees which is non-zero.).
The smoothing of Zhu and Eble can be modified to smooth an image at an angle (e.g. horizontal) with the vertical vector in an image of the physical environment. Vertical and horizontal angles are non-zero.
The motivation for the combination is provided by Tam “ the fact that the human visual system is more attuned to horizontal disparities than vertical disparities”.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
12. The method of claim 10, wherein the angle is different than a second angle between a vertical vector in an image space of a second image of the physical environment and a second smoothing direction of the second image of the physical environment ("vertical” and “horizontal” orientation shown in Table 1 of Tam [0110] where the horizontal is 90 degrees which is different than a vertical angle.).
The smoothing of Zhu and Eble can be modified to smooth an image at an angle (e.g. horizontal) with the vertical vector in an image of the physical environment. Vertical and horizontal angles different.
The motivation for the combination is provided by Tam “ the fact that the human visual system is more attuned to horizontal disparities than vertical disparities”.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al (US 2019/0101758 A1) “Zhu” in view of Eble et al. (US 10,832,427 B1) “Eble” and further in view of Noda (US 8,744,210 B2) “Noda”.
20. The non-transitory computer-readable memory of claim 20 is analyzed in view of the "non-volatile mass storage such as physical storage media." [0043] of Zhu and further in view of claim 1 where a first smoothing direction can correspond to a smoothing direction for a left eye of a Head Mounted Display and a second smoothing direction can correspond to a smoothing direction for a right eye of a Head Mounted Display (Head Mounted Displays in "120" [Figure 1] of Eble; [Fig. 2] of Zhu).
Zhu and Eble do not explicitly teach: where the first and second smoothing directions are different.
However, Noda teaches: where the first and second smoothing directions are different. (" smoothing by applying low-pass filter to pixels along a first direction" [Claim 13]. “smoothing processing … in a second direction different from the first direction” [Claim 15].) .
The smoothing of Zhu and Eble can be modified by Noda to smooth an image for a left eye and smooth an image for a left eye in a different direction.
The motivation for the combination is based on the left eye and the right eye having different viewing angles for a subject. For improved image performance, smoothing should be related to the angle of the object from each of the left eye and right eye having different vantage points.
Therefore, the Applicant’s claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention and the claim is rejected.
Allowable Subject Matter
If the 35 USC 101 abstract idea rejection is overcome:
Then Claims 16-19 are allowed.
The closest reference of record is Zhu In the Applicant’s independent claim 16 and dependent claims 17-19, the reference of Zhu does not teach: rotate an anisotropic filter kernel by an angle based on a vector that is independent of an orientation of the device; filter the depth map using the rotated anisotropic filter kernel. Zhu fails to directly anticipate or render the above underlined limitations obvious (to be used with other claimed limitations).
If the 35 USC 101 abstract idea rejection is overcome:
Then Claim 7-9 and 13-15; are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In claims 7-9 Zhu does not teach: the smoothing direction includes determining a projection of the world-fixed vector into an image space of the image of the physical environment.
In claims 13-15 Zhu does not teach: smoothing the depth map includes applying an anisotropic filter to the depth map.
Relevant Prior Art
US 8,599,403 B2
Abstract
The invention relates to a method for acquiring a substantially complete depth map from a 3-D scene. Both depth values and derivates of depth values may be used to calculate a pixel dense depth map with the steps of acquiring partial depth map from said 3-D scene, acquiring derivates of depth information from said scene, and extending said partial depth map by adding non-relevant information to said partial depth map, creating a pixel dense full depth map being spatially consistent with both said partial depth map and said derivates of depth information.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TED W BARNES whose telephone number is (571) 270-1785. The examiner can normally be reached on Mon-Fri. 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached on 571-270-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TED W. BARNES/ Ph.D. Electrical Engineering
Primary Examiner
Art Unit 2682
/TED W BARNES/Primary Examiner, Art Unit 2682