DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a real space information acquisition unit”, “a virtual space information acquisition unit”, “a user behavior determination unit”, and “a virtual space providing unit” in claims 1-6 and “a position information acquisition unit” in claim 5.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5 and 7 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as anticipated by or, in the alternative, under 35 U.S.C. 103 as obvious over Ohashi et al. (Pub No. US 20210174601 A1).
As per claim 1, Ohashi teaches the claimed:
A providing apparatus comprising:
a real space information acquisition unit configured to acquire position information on a real object present in a real space in which a user is present (Please see Ohashi [0059] ”The shape information and depth information of an object in the real world can be acquired by three-dimensional (3D) scanning a real-world space. For example, depth information of the real space can be acquired using a depth sensor employing a method such as an infrared pattern, Structured Light, or time of flight (TOF). Alternatively, depth information of the real space can be acquired from parallax information of a stereo camera”).
In this passage, the depth and shape information of an object is identified using the device 100 in figure 1 in relation to the user’s position information. Ohashi [0029] further teaches how the user’s position information is acquired “Position information of the head of the user wearing the head-mounted display 100 and orientation information such as the rotation angle and inclination of the head can be measured by a gyro sensor, an acceleration sensor, and the like incorporated in or externally attached to the head-mounted display 100”);
a virtual space information acquisition unit configured to acquire position information on a virtual object present in a virtual space (Ohashi [0062]“According to the viewpoint position and line-of-sight direction of the user set by the viewpoint and line-of-sight setting section 220, the rendering section 232 renders a virtual space object that is visible in the line-of-sight direction from the viewpoint position of the user wearing the head-mounted display 100. The rendering section 232 then gives the virtual space object to the AR generation section 234”);
a user behavior determination unit configured to determine whether the user has performed a predetermined behavior with respect to the real object in the real space (Ohashi [0080 – 0081] “FIG. 6A illustrates the coffee cup 510 on the table 530 being about to be picked up by the user who is stretching out his/her hand to the coffee cup 510. Since the user's hand has not contacted the coffee cup 510 yet, the real-world coffee cup 510 is visible as it is on the screen FIG. 6B illustrates an exemplary screen when the user's hand has contacted the coffee cup 510 of FIG. 6A. When the user's hand has contacted the coffee cup 510, the entire coffee cup 510 is specified as the partial space and a region corresponding to the coffee cup 510 viewed from the viewpoint of the user becomes the AR region”
In this passage, Ohashi predicts that the user will touch the real life coffee cup (element 510 in figure 6A). In 6B, the device has identified that the user has indeed touched the real life coffee cup, and has responded by replacing the real life coffee cup 510 with the virtual object 512.); and
a virtual space providing unit configured to make the position information in the real space and the position information in the virtual space consistent (Please refer to Ohashi Fig. 6A and 6B. The position of element 510 in relation to the position of 500 mirrors the position of element 512 in relation to the position of 500),
to generate a transition video that indicates a state between the real space and the virtual space when the user behavior determination unit determines that the user has performed the predetermined behavior with respect to the real object in the real space (Ohashi [0074] “FIG. 5A is a diagram for describing a screen displayed on the display panel 32 of the head-mounted display 100. Here, the head-mounted display 100 is of a video-transmissive type and displays a video of the outside world shot by the camera mounted in the head-mounted display 100”
Based on this passage, Figures 5A-6B are views from the video being recorded by the AR device. Figures 6A and 6B show the video transition from when the user picks up the real coffee cup 510 to the virtual object 512),
and to provide a transition space that includes the transition video before providing the virtual space to the user (Ohashi [0074-0077] “Here, the head-mounted display 100 is of a video-transmissive type and displays a video of the outside world shot by the camera mounted in the head-mounted display 100 […] When the user's hand has contacted the upper surface of the table 530 of FIG. 5A, the entire surface of the table 530 including the contacted portion is specified as the partial space, and a region corresponding to this partial space viewed from the viewpoint of the user becomes the AR region […] In this manner, only the partial space including the portion of the real space, the portion having been contacted by the body part of the user, becomes an augmented reality image on the screen. Accordingly, it is possible to reduce the feeling of discontinuity and strangeness at the time of transition from the camera image to the augmented reality image, allowing the user to have the feeling of physical presence”
In this passage, Ohashi teaches the step-by-step transition from the outside world shot by the camera to the augmented reality image. When the user’s hand touches the surface of the table, it triggers the transition from the real world view to the virtual view.
Ohashi [0083] “In the above examples, an AR effect is applied by changing the AR region into a CG image. Alternatively, a video see-through image to which a visual effect filter is applied may be displayed in the AR region” and Ohashi [0084] “Many variations of effects can also be considered. Examples of effects include generation of particles, application of a visual effect filter, a change of luminance and color, a lens effect, zoom-in/out display, generation of a sound effect, and a change of volume of background music or a sound effect”
Given that Ohashi mentions “transition from the camera image to the augmented reality image” and “Many variations of effects can also be considered”, it is obvious that this transition and effects is similar to the claimed “transition space” that includes a transition video. This also occurs before providing the virtual space to the user because the virtual space is part of the display of virtual objects in the AR region. The transition space is part of the effects that enable the transition into the virtual space with the display of virtual objects in the AR region. Thus, it would have been obvious to modify Ohashi by incorporating the transition and effects from Ohashi into the coffee cup example shown in figures 6A-6C in Ohashi in order to “reduce the feeling of discontinuity and strangeness at the time of transition from the camera image to the augmented reality image, allowing the user to have the feeling of physical presence” (Ohashi in [0077])). This modification is achieved by playing a transition video, e.g. effects during the transition from the real world camera image into the augmented reality image (that would then include a virtual space with virtual objects displayed such as virtual object 512 in figure 6B).
As per claim 2, Ohashi teaches the claimed:
The providing apparatus according to claim 1, wherein the virtual space providing unit is further configured to generate the transition video in accordance with a type of the real object (Please see Ohashi [0081-0082] and figures 6A and 6B in combination with the explanation in claim 1).
As per claim 3, Ohashi teaches the claimed:
The providing apparatus according to claim 1, wherein the predetermined behavior includes at least one of a behavior of looking at the real object and a behavior of holding his/her hand over the real object (Ohashi [0081] “FIG. 6B illustrates an exemplary screen when the user's hand has contacted the coffee cup 510 of FIG. 6A. When the user's hand has contacted the coffee cup 510, the entire coffee cup 510 is specified as the partial space and a region corresponding to the coffee cup 510 viewed from the viewpoint of the user becomes the AR region. The AR generation section 234 draws a virtual object 512 by replacing the coffee cup 510 with the virtual object 512”).
As per claim 5, Ohashi teaches the claimed:
The providing apparatus according to claim 1, further comprising:
a position information acquisition unit configured to acquire user position information that is information indicating a position of the user in the virtual space (Ohashi [0042] “An orientation sensor 64 detects position information of the head-mounted display 100 and orientation information such as the rotation angle and inclination of the head-mounted display 100. The orientation sensor 64 is implemented by appropriately combining a gyro sensor, an acceleration sensor, an angular acceleration sensor, and the like. The forward and rearward, rightward and leftward, and upward and downward movements of the head of the user may be detected by using a motion sensor implemented by combining at least one or more of a three-axis magnetic field sensor, a three-axis acceleration sensor, and a three-axis gyro (angular velocity) sensor”
In this passage, the orientation sensor corresponds to the position information acquisition unit in the claimed invention ), wherein
the position information acquisition unit is further configured to acquire the user position information such that the position information on the virtual object with respect to the user in the virtual space corresponds to the position information on the real object with respect to the user in the real space (Ohashi [0059] “The shape information and depth information of an object in the real world can be acquired by three-dimensional (3D) scanning a real-world space. For example, depth information of the real space can be acquired using a depth sensor employing a method such as an infrared pattern, Structured Light, or time of flight (TOF). Alternatively, depth information of the real space can be acquired from parallax information of a stereo camera”
Once the shape information and depth information of the real object is acquired, Ohashi in [0065] teaches that the post-processing section adjusts the virtual image to align with the depth and shape information of the real object in the real world “The post-processing section 236 performs post-processing such as depth-of-field adjustment, tone mapping, and anti-aliasing on the augmented reality image in such a manner that the augmented reality image obtained by superimposing the virtual object on the image of the real space looks natural and smooth. Further, the post-processing section 236 may apply various kinds of effects to the AR region specified by the AR region determination section 248. The post-processing section 236 may simply apply an effect to the camera image in the AR region without a CG image being superimposed or drawn for replacement in the AR region by the AR generation section 234.”).
As per claim 7, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Ohashi in view of Clement et al. (Pub No. US 20160313790 A1).
As per claim 4, Ohashi teaches the claimed:
The providing apparatus according to claim 1, wherein
the user behavior determination unit is further configured to determine whether the user has performed the predetermined behavior on the virtual object in the virtual space (Ohashi [0093] “FIG. 8A illustrates an exemplary screen when the user has clapped both hands. The user wears the tracker 500 on each of the hands. When the position of the right hand detected by the tracker 500 of the right hand and the position of the left hand detected by the tracker 500 of the left hand have come close to a predetermined distance or less, the contact determination section 247 determines that both hands have contacted with each other”
In this passage, the tracker 500 is the behavior determination unit. Ohashi teaches how it is used to determine whether the user has performed the predetermined behavior or not, using the tracker 500.),
Ohashi alone does not explicitly teach the remaining claim limitations.
However, Ohashi in combination with Clement teaches the claimed:
the virtual space providing unit is further configured to provide the transition video before providing the real space to the user when the user behavior determination unit determines that the user has performed the predetermined behavior on the virtual object in the virtual space (Clement [0029] “As noted above, while at position 4, the tracking device 310 may still track movement of the user A in the space 300 and translate the tracked physical movement into movement in the virtual world. However, the user's proximity to the edge of the tracking area 320, and the movement of a portion of the zone Z close to, or beyond, the boundary of the tracking area 320, may trigger the initiation of a fluid, or graceful, transition process from the virtual world to the real world before tracking is fully lost and real world physical movement can no longer be translated into movement in the virtual world. This may avoid the discomfort and disorientation that may be associated with a more abrupt end to the virtual world as the user A continues to move in the real world”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the tracking device as taught by Clement with the system of Ohashi in order to initiate a transition automatically when triggered by the user’s behavior. Doing so allows the device to create a smooth transition space from the virtual world back to the real world in order to avoid the feeling of discomfort the user may feel from a more abrupt transition. This transition is triggered by the user’s behavior exceeds outside of the boundary in which the AR/VR feature is capable. Doing so maintains the user’s safety within the real-world environment.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Ohashi in view of Suzuki et al. (Pub No. US 20050285879 A1).
As per claim 6, Ohashi teaches the claimed:
The providing apparatus according to claim 1, wherein
the transition video is a video that includes a part of a video of the real space and a part of a video of the virtual space (Ohashi [0077] “In this manner, only the partial space including the portion of the real space, the portion having been contacted by the body part of the user, becomes an augmented reality image on the screen. Accordingly, it is possible to reduce the feeling of discontinuity and strangeness at the time of transition from the camera image to the augmented reality image, allowing the user to have the feeling of physical presence”)
Ohashi alone does not explicitly teach the remaining claim limitations.
However, Ohashi in combination with Suzuki teaches the claimed:
the virtual space providing unit is further configured to output, as the transition video, a video in which the virtual object that has a same size and a same shape as those of the real object is superimposed on the real object (Suzuki [0072] “the image of the virtual model to be superimposed on the physical model is rendered in the memory such as the video buffer”
Suzuki [0076] “According to the first embodiment of the present invention, as described above, when a physical model and a virtual model which are identical to each other in terms of the shape and the size, it is possible to easily adjust the position/orientation of the virtual model such that the position/orientation of the virtual model exactly coincides with the position/orientation of the physical model”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the process of outputting a mixed reality image as taught by Suzuki with the system of Ohashi in order to display an accurate representation of the real objects in the virtual space. Doing so allows the observer to enter a simulation that is closely representative of the real world, by having the virtual objects mirror the real objects in size and shape, allowing the observer to have a more seamless transition into the virtual world.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SARAH Y. LEE whose telephone number is (571)272-8374. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SARAH Y. LEE
Examiner
Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
Prosecution Insights