DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20 are currently pending in the present application, with claims 1, 14, and 18 being independent.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10 January 2025 has been considered by the examiner.
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required: Claims 3 and 4 recite “an operational display frame rate” from a first or second frame rate, past the transitional display frame rate, and to the second/first frame rate respectively. The originally filed disclosure does not appear to describe an operational display frame rate separate from the frame rate and the operational frame rate past the transitional display frame rate.
Claim Objections
Claim 19 is objected to because of the following informalities: Claim 19 appears as though it should recite in part “wherein the transitional frame rate increases . Appropriate correction is required.
Claim Interpretation
First mode in which first content is displayed on the display without displaying real-world content from the camera is being interpreted as a virtual reality mode described throughout the specification. Second mode in which real-world content from the camera with non-real-world content overlaid is displayed is being interpreted as a mixed reality type mode described throughout the specification.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-19 of U.S. Patent No. 10,748,340.
Current Application
US Patent 10,748,340
1. An electronic device comprising:
a display;
a camera; and
control circuitry configured to transition the display and the camera between a first mode in which first content is displayed on the display without displaying real-world content from the camera and a second mode in which second content is displayed on the display that includes real-world content from the camera and overlaid non-real-world content, wherein the control circuitry is configured to transition the display between the first mode and the second mode by operating the display with a transitional display frame rate
1. An electronic device, comprising:
a display having an adjustable display frame rate;
a camera having an adjustable camera frame rate; and
control circuitry configured to transition the display and camera between:
a first mode in which first content is displayed on the display without displaying real-world content from the camera and in which the camera has a first camera frame rate and the display has a first display frame rate, and
a second mode in which second content is displayed on the display that includes real-world content from the camera and overlaid non-real-world content and in which the camera has a second camera frame rate and the display has a second display frame rate, wherein the first camera frame rate is a non-zero frame rate and is less than the second camera frame rate.
14. An electronic device comprising:
a display; and
control circuitry coupled to the display and configured to:
operate the display to display first content at a first frame rate in a virtual reality mode,
operate the display to display second content at a second frame rate in a mixed reality mode, and
operate the display to display third content at a transitional frame rate between the first and second frame rates when transitioning between the virtual reality mode and the mixed reality mode.
11. An electronic device, comprising:
a display having an adjustable display frame rate;
a camera having an adjustable camera frame rate; and
control circuitry configured to transition the display and camera between:
a first mode in which first content is displayed without displaying real-world content from the camera and the display has a first display frame rate, and
a second mode in which second content is displayed and the display has a second display frame rate that is different from the first display frame rate, wherein the second content includes real-world content from the camera and overlaid non-real-world content, wherein the control circuitry is configured to control the display to exhibit a transitional display frame rate when transitioning the display between the first mode and the second mode, wherein the control circuitry is configured to reconfigure the camera to exhibit a transitional camera frame rate when transitioning the display between the first mode and the second mode, wherein the transitional camera frame rate corresponds to the transitional display frame rate, and wherein the transitional display frame rate is between the first display frame rate and the second display frame rate.
18. An electronic device comprising:
a display; and
control circuitry coupled to the display and configured to:
operate the display to display content at a frame rate in a mixed reality mode, and operate the display to transition out of the mixed reality mode by operating the display with a transitional frame rate that varied across a transition time period.
11. An electronic device, comprising:
a display having an adjustable display frame rate;
a camera having an adjustable camera frame rate; and
control circuitry configured to transition the display and camera between:
a first mode in which first content is displayed without displaying real-world content from the camera and the display has a first display frame rate, and
a second mode in which second content is displayed and the display has a second display frame rate that is different from the first display frame rate, wherein the second content includes real-world content from the camera and overlaid non-real-world content, wherein the control circuitry is configured to control the display to exhibit a transitional display frame rate when transitioning the display between the first mode and the second mode, wherein the control circuitry is configured to reconfigure the camera to exhibit a transitional camera frame rate when transitioning the display between the first mode and the second mode, wherein the transitional camera frame rate corresponds to the transitional display frame rate, and wherein the transitional display frame rate is between the first display frame rate and the second display frame rate.
Although the claims at issue are not identical, they are not patentably distinct from each other because independent claims in the current application are an obvious variant of the claims in the US Patent. The other minor differences between the claims do not appear to change the scope of the claimed invention.
Similar mapping can be performed between the dependent claims in the current application and dependent claims in the US Patent.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 12,198,276.
Current Application
US Patent 12,198,276
1. An electronic device comprising:
a display;
a camera; and
control circuitry configured to transition the display and the camera between a first mode in which first content is displayed on the display without displaying real-world content from the camera and a second mode in which second content is displayed on the display that includes real-world content from the camera and overlaid non-real-world content, wherein the control circuitry is configured to transition the display between the first mode and the second mode by operating the display with a transitional display frame rate
1. An electronic device comprising:
a display;
a camera; and
control circuitry configured to:
control the display to exhibit a transitional display frame rate when transitioning between a first mode in which virtual reality content is displayed on the display and a second mode in which mixed reality content is displayed on the display using a display frame rate that is less than the transitional display frame rate, and control the camera to operate at a camera frame rate synchronized with the transitional display frame rate when the display exhibits the transitional display frame rate.
14. An electronic device comprising:
a display; and
control circuitry coupled to the display and configured to:
operate the display to display first content at a first frame rate in a virtual reality mode,
operate the display to display second content at a second frame rate in a mixed reality mode, and
operate the display to display third content at a transitional frame rate between the first and second frame rates when transitioning between the virtual reality mode and the mixed reality mode.
1. An electronic device comprising:
a display;
a camera; and
control circuitry configured to:
control the display to exhibit a transitional display frame rate when transitioning between a first mode in which virtual reality content is displayed on the display and a second mode in which mixed reality content is displayed on the display using a display frame rate that is less than the transitional display frame rate, and control the camera to operate at a camera frame rate synchronized with the transitional display frame rate when the display exhibits the transitional display frame rate.
18. An electronic device comprising:
a display; and
control circuitry coupled to the display and configured to:
operate the display to display content at a frame rate in a mixed reality mode, and operate the display to transition out of the mixed reality mode by operating the display with a transitional frame rate that varied across a transition time period.
15. An electronic device comprising:
a display; and
control circuitry coupled to the display and configured to:
operate the display to display content at a frame rate in a virtual reality mode, and operate the display to transition out of the virtual reality mode by varying the frame rate, at which the content is displayed during the transition, by a constant adjustment rate across a transition time period for the transition.
Although the claims at issue are not identical, they are not patentably distinct from each other because independent claims in the current application are an obvious variant of the claims in the US Patent. The other minor differences between the claims do not appear to change the scope of the claimed invention.
Similar mapping can be performed between the dependent claims in the current application and dependent claims in the US Patent.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-5, 7, 11, and 17-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to claims 3 and 4, given the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed terms is unclear. For instance, as drafted, it is not immediately clear as to what constitutes the operational frame rate, how the operational display frame is from the first/second display frame rate, and goes to the second/first respective frame rate. The disclosure does not appear to describe the current claim language beyond the words themselves (see specification object). The examiner respectfully requests the applicant clarify the scope of the claimed limitation.
With respect to claim 5, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, as currently drafted it is not immediately clear as to what is meant by incrementally varying the transitional display frame rate. How is it incrementally varied – is it constant is some spots and varied in others? Does the variation rate change? The originally filed disclosure supports varying the frame rate in the manner illustrated in Fig. 7. Paragraph 31 discusses gradual transitions in frame rates when switching between virtual reality and mixed reality modes. However, it is unclear if applicant is claiming any pattern of variation between modes. The examiner respectfully requests the applicant clarify the scope of the claimed limitation. For the purposes of further examination, the examiner is interpreting the claimed limitation along the lines of fig. 7.
With respect to claim 7, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, as currently drafted it is not immediately clear as to what is meant by the transitional camera frame rate being based on the transitional display frame rate. The originally filed disclosure discusses synchronizing the camera frame rate to the display frame rate to avoid latency issues, see for instance, paragraph 34 and fig. 7. However, synchronizing the camera and display frame rates does not clarify the scope of the transitional camera frame rate being based on the transitional display frame rate. The specification further supports setting the camera frame rate to the display frame rate, see for instance, paragraphs 28 and 34 and fig. 7 The examiner respectfully requests the applicant clarify the scope of the claimed limitation. For the purposes of further examination, the examiner is interpreting the claim to synchronize the camera frame rate to the display frame rate when transitioning between modes as discussed in paragraph 34 and illustrated in fig. 7.
With respect to claim 10, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, as currently drafted it is not immediately clear as to what is meant by without suing the transitional camera frame rate when transitioning the camera from the second mode to the first mode. The specification in paragraph 35 and fig. 7 discuss reducing the camera frame rate to 30 Hz at t5. However, it is unclear as to how the camera is operated in the negative limitation of without using the transitional camera frame rate when transitioning the camera from the second mode to the first mode. The examiner respectfully requests the applicant clarify the scope of the claimed limitation.
With respect to claim 11, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, as currently drafted it is not immediately clear as to what is meant by varying the transitional camera frame rate. How is it varied – is it constant is some spots and varied in others? Does the variation rate change? The originally filed disclosure supports varying the frame rate in the manner illustrated in Fig. 7. Paragraph 31 discusses gradual transitions in frame rates when switching between virtual reality and mixed reality modes. However, it is unclear if applicant is claiming any pattern of variation between modes. The examiner respectfully requests the applicant clarify the scope of the claimed limitation. For the purposes of further examination, the examiner is interpreting the claimed limitation along the lines of fig. 7.
With respect to claim 17, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitation is unclear. For instance, as currently drafted it is not immediately clear as to what is meant by varying the transitional frame rate incrementally. How is it varied – is it constant is some spots and varied in others? Does the variation rate change? The originally filed disclosure supports varying the frame rate in the manner illustrated in Fig. 7. Paragraph 31 discusses gradual transitions in frame rates when switching between virtual reality and mixed reality modes. However, it is unclear if applicant is claiming any pattern of variation between modes. The examiner respectfully requests the applicant clarify the scope of the claimed limitation. For the purposes of further examination, the examiner is interpreting the claimed limitation along the lines of fig. 7.
With respect to claim 18, when afforded the broadest reasonable interpretation in light of the corresponding disclosure, the scope of the claimed limitations is unclear. For instance, as currently drafted it is not immediately clear as to what is meant by a transitional frame rate that varied across a transition time period. How is it varied – is it constant is some spots and varied in others? Does the variation rate change? The originally filed disclosure supports varying the frame rate in the manner illustrated in Fig. 7. Paragraph 31 discusses gradual transitions in frame rates when switching between virtual reality and mixed reality modes. However, it is unclear if applicant is claiming any pattern of variation between modes. The examiner respectfully requests the applicant clarify the scope of the claimed limitation. For the purposes of further examination, the examiner is interpreting the claimed limitation along the lines of fig. 7. The examiner is further interpreting the past tense “varied” as being in the present tense, such that the operation out of the mixed reality mode is done by operating the display with a transitional frame rate that varies across a transitional time period.
Claims depending thereon do not cure the noted deficiencies and are accordingly also rejected using substantially similar rationale as to that set forth for the claims from which they depend.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 9 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 6 recites “wherein the control circuitry is configured to transition the camera between the first mode and the second mode by operating the camera with a transitional camera frame rate”. Claim 9, which depends from claim 6 recites “wherein the control circuitry is configured to operate the camera with the transitional camera frame rate when transitioning the camera from the first mode to the second mode”. Claim 9 appears to be a reworded version of claim 6 and thus does not appear to further limit the limitation over claim 6. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 6, 9, and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee (US PG Publication 2016/0253843) in view of Sra et al. (“Procedurally Generated Virtual Reality from 3D Reconstructed Physical Space,” 2016) in further view of Fang et al. (“Real-Time Motion Tracking for Mobile Augmented/Virtual Reality Using Adaptive Visual-Inertial Fusion”, 2017).
Regarding claim 1, Lee teaches an electronic device comprising: a display; a camera; and control circuitry configured (see for instance, paragraphs 23 and 41-43 and fig. 3) to transition the display and the camera between a first mode in which first content is displayed on the display without displaying real-world content from the camera and a second mode in which second content is displayed on the display that includes real-world content from the camera and overlaid non-real-world content , wherein the control circuitry is configured to transition the display between the first mode and the second mode by operating the display with a transitional display frame rate (A VR technology renders a three-dimensional space images through computer simulation…The electronic display is used to display the VR images, see paragraph 23. To display the AR images, reference is made to the flow chart shown in Fig. 3…in the AR mode, the electronic the electronic display device activates its camera to capture images of a real space, see paragraph 42. The electronic display device can also be operated in an AR mode that utilizes the augmented reality technology to display the virtual object within a real space, see paragraph 23 Note: The transitional frame rate can be one of the display frame rates for the VR/AR modes as currently claimed).
While Lee teaches the broadest reasonable interpretation of claim 1, Lee is silent with respect to display frame rates being different between AR and VR and thus Sra and Fang are being incorporated into the rejection to teach different modes can have different frame rates.
In the same art of modified reality, Sra teaches that depth data is used for tracking objects during the VR experience, see abstract. The forward facing camera tracks objects during VR mode, see for instance, page 195, LHC, paragraph 3, page 196, fig. 8, and page 197, section 6.1, paragraph 2. For acquiring input data we use the tango device which features a motion and depth sensing camera to create a 3D map of the environment. The integrated sensors continuously return the 3D position and orientation of the device producing a registered point cloud of the environment in real-time, see page 194, LHC, paragraph 3. A point cloud created through any other scanning technique or device would also be valid input for our system, see page 194, LHC, paragraph 3. We focus on detecting open space and use boundary elements to create a visual barrier between the detected open space and the obstacles like furniture or walls….The rest of point P belong to potential obstacles…Objects that are above the user’s head, like the ceiling, do not impact the walkable area and are ignored since there is no possibility of collision with them, see page 194, LHC, paragraphs 1-4 and fig. 5.
It would have been obvious to one of ordinary skill in the art having the teachings of Lee and Sra in front of them before the effective filing date of the claimed invention to incorporate 3D virtual reality construction as taught by Sra into Lee’s AR/VR graphics system, as creating 3D map of the environment from motion and depth sensing cameras, determining obstacles and detecting an open space free of obstacles, such as described by Sra was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Lee.
The modification of Lee with Sra would have explicitly allowed the camera to operate with a camera frame rate while operating in the first mode.
The motivation for combining Lee with Sra would have been to increase user comfort and maneuverability, explicitly set up a safe region and increase system and design flexibility.
Lee in view of Sra are silent with respect to display frame rates being different between AR and VR.
In the same art of modified reality, Fang teaches that for “mobile AR, the frequency of arriving poses should reach the normal video frame-rate (25 Hz), which is usually defined as a standard regulation for a real-time performance. This standard is also considered in this paper for real-time performances other than in mobile VR. Because the real-time performance in mobile VR is superior to conventional scenarios, the arriving frequency of the 6-DoF motion tracking should reach 60 Hz or more. Only in this way, the participant within the VR environment can enjoy a comfortable experience. Otherwise, the delay phenomenon would cause the user disgust in VR environments,” see page 2, paragraph 5.
It would have been obvious to one of ordinary skill in the art having the teachings of Lee, Sra, and Fang in front of them before the effective filing date of the claimed invention to incorporate different real-time display frame rates and motion tracking as taught by Fang into Lee’s AR/VR graphics system, as having an increased frame rate for VR to improve user comfort, such as described by Fang was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Lee and Sra.
The modification of Lee and Sra with Fang would have explicitly allowed the second display frame rate to be less than the first display frame rate.
The motivation for combining Lee, and Sra with Fang would have been to increase user comfort and maneuverability.
Regarding claim 2, Lee in view of Sra in further view of Fang teach the electronic device defined in claim 1 and further teach wherein the first content is displayed on the display using a first display frame rate and wherein the second content is displayed on the display using a second display frame rate less than the first display frame rate (A VR technology renders a three-dimensional space images through computer simulation…The electronic display is used to display the VR images, see Lee, paragraph 23. To display the AR images, reference is made to the flow chart shown in Fig. 3…in the AR mode, the electronic the electronic display device activates its camera to capture images of a real space, see Lee, paragraph 42. The electronic display device can also be operated in an AR mode that utilizes the augmented reality technology to display the virtual object within a real space, see Lee, paragraph 23 Fang teaches that for “mobile AR, the frequency of arriving poses should reach the normal video frame-rate (25 Hz), which is usually defined as a standard regulation for a real-time performance. This standard is also considered in this paper for real-time performances other than in mobile VR. Because the real-time performance in mobile VR is superior to conventional scenarios, the arriving frequency of the 6-DoF motion tracking should reach 60 Hz or more. Only in this way, the participant within the VR environment can enjoy a comfortable experience. Otherwise, the delay phenomenon would cause the user disgust in VR environments,” see page 2, paragraph 5). The motivation to combine Lee, Sra, and Fang is the same as that which was set forth in claim 1.
Regarding claim 6, , Lee in view of Sra in further view of Fang teach the electronic device defined in claim 1 and further teach wherein the control circuitry is configured to transition the camera between the first mode and the second mode by operating the camera with a transitional camera frame rate (A VR technology renders a three-dimensional space images through computer simulation…The electronic display is used to display the VR images, see Lee, paragraph 23. To display the AR images, reference is made to the flow chart shown in Fig. 3…in the AR mode, the electronic the electronic display device activates its camera to capture images of a real space, see Lee, paragraph 42. The electronic display device can also be operated in an AR mode that utilizes the augmented reality technology to display the virtual object within a real space, see Lee, paragraph 23. Depth data is used for tracking objects during the VR experience, see Sra, abstract. The forward facing camera tracks objects during VR mode, see for instance, Sra, page 195, LHC, paragraph 3, page 196, fig. 8, and page 197, section 6.1, paragraph 2. For acquiring input data we use the tango device which features a motion and depth sensing camera to create a 3D map of the environment. The integrated sensors continuously return the 3D position and orientation of the device producing a registered point cloud of the environment in real-time, see Sra, page 194, LHC, paragraph 3. A point cloud created through any other scanning technique or device would also be valid input for our system, see Sra, page 194, LHC, paragraph 3. We focus on detecting open space and use boundary elements to create a visual barrier between the detected open space and the obstacles like furniture or walls….The rest of point P belong to potential obstacles…Objects that are above the user’s head, like the ceiling, do not impact the walkable area and are ignored since there is no possibility of collision with them, see Sra, page 194, LHC, paragraphs 1-4 and fig. 5). The motivation to combine Lee, Sra, and Fang is the same as that which was set forth in claim 1.
Regarding claim 9, Lee in view of Sra in further view of Fang teach the electronic device defined in claim 6 and further teach wherein the control circuitry is configured to operate the camera with the transitional camera frame rate when transitioning the camera from the first mode to the second mode (A VR technology renders a three-dimensional space images through computer simulation…The electronic display is used to display the VR images, see Lee, paragraph 23. To display the AR images, reference is made to the flow chart shown in Fig. 3…in the AR mode, the electronic the electronic display device activates its camera to capture images of a real space, see Lee, paragraph 42. The electronic display device can also be operated in an AR mode that utilizes the augmented reality technology to display the virtual object within a real space, see Lee, paragraph 23. Depth data is used for tracking objects during the VR experience, see Sra, abstract. The forward facing camera tracks objects during VR mode, see for instance, Sra, page 195, LHC, paragraph 3, page 196, fig. 8, and page 197, section 6.1, paragraph 2. For acquiring input data we use the tango device which features a motion and depth sensing camera to create a 3D map of the environment. The integrated sensors continuously return the 3D position and orientation of the device producing a registered point cloud of the environment in real-time, see Sra, page 194, LHC, paragraph 3. A point cloud created through any other scanning technique or device would also be valid input for our system, see Sra, page 194, LHC, paragraph 3. We focus on detecting open space and use boundary elements to create a visual barrier between the detected open space and the obstacles like furniture or walls….The rest of point P belong to potential obstacles…Objects that are above the user’s head, like the ceiling, do not impact the walkable area and are ignored since there is no possibility of collision with them, see Sra, page 194, LHC, paragraphs 1-4 and fig. 5). The motivation to combine Lee, Sra, and Fang is the same as that which was set forth in claim 1.
Regarding claim 10, Lee in view of Sra in further view of Fang teach the electronic device defined in claim 9 and further teach wherein the control circuitry is configured to operate the camera without using the transitional camera frame rate when transitioning the camera from the second mode to the first mode (A VR technology renders a three-dimensional space images through computer simulation…The electronic display is used to display the VR images, see Lee, paragraph 23. To display the AR images, reference is made to the flow chart shown in Fig. 3…in the AR mode, the electronic the electronic display device activates its camera to capture images of a real space, see Lee, paragraph 42. The electronic display device can also be operated in an AR mode that utilizes the augmented reality technology to display the virtual object within a real space, see Lee, paragraph 23. Depth data is used for tracking objects during the VR experience, see Sra, abstract. The forward facing camera tracks objects during VR mode, see for instance, Sra, page 195, LHC, paragraph 3, page 196, fig. 8, and page 197, section 6.1, paragraph 2. For acquiring input data we use the tango device which features a motion and depth sensing camera to create a 3D map of the environment. The integrated sensors continuously return the 3D position and orientation of the device producing a registered point cloud of the environment in real-time, see Sra, page 194, LHC, paragraph 3. A point cloud created through any other scanning technique or device would also be valid input for our system, see Sra, page 194, LHC, paragraph 3. We focus on detecting open space and use boundary elements to create a visual barrier between the detected open space and the obstacles like furniture or walls….The rest of point P belong to potential obstacles…Objects that are above the user’s head, like the ceiling, do not impact the walkable area and are ignored since there is no possibility of collision with them, see Sra, page 194, LHC, paragraphs 1-4 and fig. 5). The motivation to combine Lee, Sra, and Fang is the same as that which was set forth in claim 1.
Claim(s) 12 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee (US PG Publication 2016/0253843) in view of Sra et al. (“Procedurally Generated Virtual Reality from 3D Reconstructed Physical Space,” 2016) in further view of Fang et al. (“Real-Time Motion Tracking for Mobile Augmented/Virtual Reality Using Adaptive Visual-Inertial Fusion”, 2017) as applied to claim 1 above, in further view of Mammano (“Looking Through Oculus Rift,” 2015)..
Regarding claim 12, Lee in view of Sra in further view of Fang teach the electronic device defined in claim 1 and further teach a head-mounted support structure (see for instance, Lee, paragraphs 23 and 29 and fig. 1 and Sra’s discussion of the HTC Vive and Google Project Tango devices) that is configured to support the camera in an outwardly facing configuration.
Lee in view of Sra in further view of Fang do not appear to explicitly state that the HMD is configured to support the camera in an outwardly facing configuration.
In the same art of augmented reality, Mammano teaches that an HMD can be configured to support the camera in an outwardly facing configuration and a display in an inwardly facing direction, see for instance, page 1, paragraph 1, page 3, paragraphs 1-3 and figs. 1-3.
It would have been obvious to one of ordinary skill in the art at the time of the invention having the teachings of Lee, Sra, Fang, and Mammano in front of them before the effective filing date of the claimed invention to incorporate outwardly facing cameras and an inwardly facing display as taught by Mammano into Lee’s AR/VR graphics system, as having the camera face outwardly and a display face inwardly, such as described by Mammano was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Lee, Sra, and Fang.
The modification of Lee, Sra, and Fang with Mammano would have explicitly allowed the HMD to be configured to support the camera in an outwardly facing configuration.
The motivation for combining Lee, Sra, and Fang with Mammano would have been to one way to implement the camera described in Lee, and to extend VR HMDs to include AR and gesture recognition, see Mammano, page 1, paragraph 1.
Regarding claim 13, Lee in view of Sra in further view of Fang teach the electronic device defined in claim 1 and further teach a head-mounted support structure that is configured to support the camera and the display (see for instance, Lee, paragraphs 23 and 29 and fig. 1 and Sra’s discussion of the HTC Vive and Google Project Tango devices), wherein the camera faces in a first direction, and wherein the display faces in a second direction opposite to the first direction.
Lee in view of Sra in further view of Fang do not appear to explicitly state that in the HMD - the camera faces in a first direction, and the display faces in a second direction opposite to the first direction.
In the same art of augmented reality, Mammano teaches that an HMD can be configured to support the camera in an outwardly facing configuration and a display in an inwardly facing direction, see for instance, page 1, paragraph 1, page 3, paragraphs 1-3 and figs. 1-3.
It would have been obvious to one of ordinary skill in the art having the teachings of Lee, Sra, Fang, and Mammano in front of them before the effective filing date of the claimed invention to incorporate outwardly facing cameras and an inwardly facing display as taught by Mammano into Lee’s AR/VR graphics system, as having the camera face outwardly and a display face inwardly, such as described by Mammano was well known at the time of the effective filing date invention and would have yielded predictable results in combination with Lee, Sra, and Fang.
The modification of Lee, Sra, and Fang with Mammano would have explicitly allowed the HMD to be configured to support a camera in an first direction and a display in a second direction opposite to the first direction.
The motivation for combining Lee, Sra, and Fang with Mammano would have been to one way to implement the camera described in Lee, and to extend VR HMDs to include AR and gesture recognition, see Mammano, page 1, paragraph 1.
Allowable Subject Matter
Since no prior art has been applied, based on their current scope, claims 3-5, 7, 8, and 11 would be allowable if rewritten to overcome the rejection(s) set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Since no prior art has been applied, based on their current scope, claims 14-20 would be allowable if rewritten or amended to overcome the rejection(s) set forth in this Office action.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US PG Publication 2011/0234791 to Miyashita et al. teaches the information processing apparatus adjusts a frame rate and the like of an imaging apparatus such as a camera to periodically acquire a real space image, see for instance, paragraph 95.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J COBB whose telephone number is (571)270-3875. The examiner can normally be reached Monday - Friday, 11am - 7pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J COBB/Primary Examiner, Art Unit 2615
Prosecution Insights