DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the claim describes a machine-readable storage medium.
Further, Applicant's specification, at paragraph [0164], fails to explicitly define the scope of machine-readable storage medium. Thus, in giving the term its plain meaning (see MPEP 2111.01), the claimed machine-readable storage medium is considered to include data signals per se. Data signals per se are not statutory as they fail to fall into one of the four statutory categories of invention.
As an additional note, a non-transitory computer readable medium having executable programming instructions stored thereon is considered statutory as non-transitory computer readable media excludes transitory data signals.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 4-5, 15 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ramsby et al (US 20160210784 A1).
Regarding claim 1, Ramsby discloses a method (Ramsby [0053], “methods and
processes described herein”) comprising:
receiving, by one or more processors, a video that includes a depiction of one or more real-world objects in a real-world environment (Ramsby [0079], “Controller 1006 (processor) may be configured to generate or update a three-dimensional model of a surrounding environment using information (receiving a video, from image sensors, that includes a depiction of one or more real-world objects in a real-world environment) from outward facing image sensors”);
generating a three-dimensional (3D) model of the real-world environment based on the video (Ramsby [0045], “A dynamically positioned augmented reality object may be dynamically placed at an appropriate depth”; [0079], “Controller 1006 may be configured to generate … a three-dimensional model of a surrounding environment using information (video) from outward facing image sensors”);
determining, based on the 3D model of the real-world environment, that an augmented reality (AR) item has been placed in the video at a particular 3D position (Ramsby [0047], “an augmented reality object may be placed at desired apparent real world depth”);
identifying a portion of the 3D model corresponding to the real-world environment currently being displayed on a screen (Ramsby [0027], “a portion of the bounding region of the augmented reality object remains in the FOV”);
determining that the 3D position of the AR item is excluded from the portion of the 3D model currently being displayed on the screen (Ramsby [0028], “if the user looks to the left, the right edge of the FOV will appear to push the augmented reality object and keep the left 50% of the width of the bounding region in the FOV.”);
detecting movement of a camera used to capture the video of a new portion of the real-world environment (Ramsby [0013], “a field of view (FOV) of a wearer of a near-eye display device changes (comprising a camera used to capture the video of a new portion of the real-world environment) … target region of interest is to be displayed when the FOV of the wearer is moving”);
updating the portion of the 3D model corresponding to the new portion of the real- world environment currently being displayed on a screen (Ramsby [0013], “the perceived real world location of selected augmented reality objects may be adjusted as a field of view (FOV) of a wearer of a near-eye display device changes.”); and
determining that the 3D position of a portion of the AR item is included in the updated portion of the 3D model (Ramsby fig. 2C; [0029], “When displayed in the FOV 206 according to the target overlap parameter 208, the entirety of the bounding region 202 is displayed.”).
Ramsby does not expressly disclose
in response to determining that the 3D position of the AR item is excluded from the portion of the 3D model displaying an indicator that identifies the 3D position of the AR item in the 3D model relative to the portion of the 3D model currently being displayed on the screen;
However, Ramsby suggests
in response to determining that the 3D position of the AR item is excluded from the portion of the 3D model, displaying an indicator that identifies the 3D position of the AR item in the 3D model relative to the portion of the 3D model currently being displayed on the screen (Ramsby fig. 2B; [0028], “The 50% (determining that the 3D position of the AR item is excluded) of the bounding region (bounding region/displaying an indicator that identifies the 3D position of the AR item) that is displayed in the x axis depends on which way the FOV is changing. For example, as the user looks to the right, the left edge of the FOV will appear to push the augmented reality object and keep the right 50% of the width of the bounding region in the FOV.”);
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to provide a bounding box around partially displayed objects. This would have been done to enable users to coordinate and adjust their views as desired.
Regarding claim 3, Ramsby discloses the method of any one of claims 1-2, further comprising:
detecting a change in orientation of a camera used to capture the video (Ramsby fig. 5; [0044], “FIG. 5 shows ... a timeline of example views 502 through an HMD, such as HMD 104 of FIG. 1, from the perspective of a user at times t.sub.0, … t.sub.3 (vertical orientation of the wall changes as the user, wearing an HMD that captures a video, moves)”); and
continuously modifying the indicator of the AR item based on the detected change (Ramsby [0044], “FIG. 5 shows another diagram 500 illustrating a timeline of example views 502 through an HMD, such as HMD 104 of FIG. 1, from the perspective of a user at times t.sub.0, t.sub.1, t.sub.2, and t.sub.3 (orientation of AR object 516, is continuously modified based on the detected change)”).
Regarding claim 4, Ramsby discloses the method of any one of claims 1-2, wherein the indicator comprises a visual representation of the AR item (Ramsby [0039], “the augmented reality object but may instead include a representation of the augmented reality object”).
Regarding claim 5, Ramsby discloses the method of any one of claims 1-2, further comprising:
determining that the 3D position of the AR item is included in the updated portion of the 3D model (Ramsby [0048], “when the user glances back and views the augmented reality object in a new, dynamically-adjusted position”).
Claim 15 recites a system which corresponds to the function performed by the method of claim 1. As such, the mapping and rejection of claim 1 above is considered applicable to the system of claim 15.
Additionally, Ramsby discloses a system (Ramsby [0032], “head-mounted display system”).
Claim 17 recites a system which corresponds to the function performed by the method of claim 3. As such, the mapping and rejection of claim 3 above is considered applicable to the system of claim 17.
Claim 18 recites a system which corresponds to the function performed by the method of claim 4. As such, the mapping and rejection of claim 4 above is considered applicable to the system of claim 18.
Claim 19 recites a system which corresponds to the function performed by the method of claim 5. As such, the mapping and rejection of claim 5 above is considered applicable to the system of claim 19.
Claim 20 recites a machine-readable storage medium which corresponds to the function performed by the method of claim 1. As such, the mapping and rejection of claim 1 above is considered applicable to the machine-readable storage medium of claim 20.
Additionally, Ramsby discloses a machine-readable storage medium (Ramsby [0057]-[0058]).
Claims 2 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ramsby in view of Molyneaux et al (US 20210042988 A1).
Regarding claim 2, Ramsby discloses the method of claim 1, further comprising:
But does not disclose obtaining depth data related to the real-world environment, wherein the 3D model is generated based on the depth data.
However, Molyneaux discloses
obtaining depth data related to the real-world environment, wherein the 3D model is generated based on the depth data ([0071], “Raw depth data may come from a depth sensor to create a depth image.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Ramsby with Molyneaux to capture real depth data using depth sensors. This would have been done to obtain accurate depth data by using depth sensors and using the data to process and utilize environment in an accurate manner.
Claim 16 recites a system which corresponds to the function performed by the method of claim 2. As such, the mapping and rejection of claim 2 above is considered applicable to the system of claim 16.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Ramsby in view of Mullins et al (US 20160049005 A1).
Regarding claim 10, Ramsby discloses the method of any one of claims 1-2, but does not disclose wherein the indicator comprises an arrow, the arrow pointing towards a direction of the 3D position.
However, Mullins discloses the indicator comprises an arrow, the arrow pointing towards a direction of the 3D position (Mullins [0054], “The visualization may include rendering a three-dimensional object (e.g., model of a beating heart), a two-dimensional object (e.g., arrow or symbols), displaying parts of the scene or objects in different colors.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Ramsby with Mullins to display directional arrows. This would have been done to provide useful objects in the virtual environment.
Allowable Subject Matter
Claims 6, 8-9 and 11-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, Ramsby discloses an indicator, however, none of the prior art of record, alone or in combination, disclose
in response to determining that the 3D position of the AR item is included in the updated portion of the 3D model:
removing the indicator from being displayed; and
Regarding claim 8, none of the prior art of record, alone or in combination, disclose the claim as recited, as a whole.
Regarding claim 9, none of the prior art of record, alone or in combination, disclose the claim as recited, as a whole.
Regarding claims 11-13, none of the prior art of record, alone or in combination, disclose the claims as recited, as a whole.
Claim 14 is allowable for depending from claim 13.
Conclusion
See the notice of references cited (PTO-892) for prior art made of record, including art that is not relied upon but considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JITESH PATEL whose telephone number is (571)270-3313. The examiner can normally be reached 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A. Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JITESH PATEL/Primary Examiner, Art Unit 2612
Prosecution Insights