DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Fukushima
Claims 1, 9, 10, 16 and 17 are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Fukushima et al.(USPubN 2008/0218515; hereinafter Fukushima).
As per claim 1, Fukushima teaches a head-mounted display comprising one or more processors and/or circuitry configured to(“This interface device employs a head-mount display system that directly displays an image before the eyes” in Para.[0005], “The three-dimensional-image display apparatus 100 includes a processor 1 such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a numeric coprocessor, and a physical calculation processor, a read only memory (ROM) 2 that stores BIOS, a random access memory (RAM) 3 that rewritably stores various kinds of data, a hard disk drive (HDD)” in Para.[0036]):
execute acquisition processing to acquire information on motion of a user(“a user interface (UI) 6 through which a user inputs various kinds of instructions to a main device and displays various kinds of information in the main device.” in Para.[0036]); and
execute control processing to control to reproduce tactile sensation with a second virtual object, which is a virtual operation body that exerts action on a first virtual object specified by the user, and is selected based on the information on the motion of the user(“the interaction calculator 13 calculates the interaction between the virtual object RV and another virtual object V so that the virtual object RV (the knife) in FIG. 32A can be expressed to cut the column-shaped virtual object V (for example, a cake), as shown in FIG. 32B. When the knife shape is displayed as the virtual object RV as explained above, preferably the cutting edge of the knife shape is displayed to correspond to the transparent portion 72 of the real object 7. Accordingly, the observer can operate to cut the cake while acquiring the feeling that the transparent portion 72 is in contact with the display surface of the three-dimensional-image display unit 5. As a result, live feeling and sense of presence of the virtual object RV can be improved while improving the operability” in Para.[0178]).
As per claim 9, Fukushima teaches wherein in the control processing, it is controlled such that a position of the second virtual object and a position at which the motion of the user was detected have a predetermined positional relationship(“the interaction calculator 13 calculates the interaction between the virtual object RV and another virtual object V so that the virtual object RV (the knife) in FIG. 32A can be expressed to cut the column-shaped virtual object V (for example, a cake), as shown in FIG. 32B. When the knife shape is displayed as the virtual object RV as explained above, preferably the cutting edge of the knife shape is displayed to correspond to the transparent portion 72 of the real object 7. Accordingly, the observer can operate to cut the cake while acquiring the feeling that the transparent portion 72 is in contact with the display surface of the three-dimensional-image display unit 5. As a result, live feeling and sense of presence of the virtual object RV can be improved while improving the operability” in Para.[0178], The position of the virtual object V can be interpreted as a position of the second virtual object and the position of the virtual object RV can be interpreted as a position at which the motion of the user was detected.).
As per claim 10, Fukushima teaches wherein in the control processing, it is controlled such that the tactile sensation with the second virtual object is reproduced even if the second virtual object is distant from a position where the motion of the user was detected(“the interaction calculator 13 calculates the interaction between the virtual object RV and another virtual object V so that the virtual object RV (the knife) in FIG. 32A can be expressed to cut the column-shaped virtual object V (for example, a cake), as shown in FIG. 32B. When the knife shape is displayed as the virtual object RV as explained above, preferably the cutting edge of the knife shape is displayed to correspond to the transparent portion 72 of the real object 7. Accordingly, the observer can operate to cut the cake while acquiring the feeling that the transparent portion 72 is in contact with the display surface of the three-dimensional-image display unit 5. As a result, live feeling and sense of presence of the virtual object RV can be improved while improving the operability” in Para.[0178], The live feeling and sense of presence of the virtual object R can be even the user’s hand doesn’t touch the virtual object R.).
As per claim 16, the limitations in the claim 16 has been discussed in the rejection claim 1 and rejected under the same rationale.
As per claim 17, Fukushima teaches a non-transitory computer readable medium that stores a program, wherein the program causes a computer to execute a method for controlling a head-mounted display(“Code modules or any type of data may be stored on any type of non-transitory computer-readable medium, such as physical computer storage including hard drives, solid state memory, random access memory (RAM), read only memory (ROM), optical disc, volatile or non-volatile storage, combinations of the same and/or the like. The methods and modules (or data) may also be transmitted as generated data signals (e.g., as part of a carrier wave or other analog or digital propagated signal) on a variety of computer-readable transmission mediums, including wireless-based and wired/cable-based mediums, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed processes or process steps may be stored, persistently or otherwise, in any type of non-transitory, tangible computer storage or may be communicated via a computer-readable transmission medium” in Para.[0277]) and the other limitations in the claim 17 has been discussed in the rejection claim 1 and rejected under the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Fukushima in view of Xu
Claims 2-4, 6 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Fukushima et al.(USPubN 2008/0218515; hereinafter Fukushima) in view of Xu et al.(USPubN 2021/0255485; hereinafter Xu).
As per claim 2, Fukushima teaches all of limitation of claim 1.
Fukushima is silent about wherein the first virtual object is specified by a line-of-sight of the user.
Xu teaches wherein the first virtual object is specified by a line-of-sight of the user(“the user may move their head or eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes may be tracked and an accommodation/vergence model may be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing. In some implementations, the wearable system may use ray casting techniques to determine which real or virtual objects are along the direction of the user's head pose or eye pose” in Para.[0133]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Xu in order to improve user experience with head-mounted display devices.
As per claim 3, Fukushima teaches all of limitation of claim 2.
Fukushima is silent about wherein the one or more processors and/or the circuitry configured to further execute selection processing to select the first virtual object, and wherein in the selection processing, the first virtual object is selected based on time during which the line-of-sight of the user is directed.
Xu teaches wherein the one or more processors and/or the circuitry configured to further execute selection processing to select the first virtual object, and wherein in the selection processing, the first virtual object is selected based on time during which the line-of-sight of the user is directed (“the user may move their head or eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes may be tracked and an accommodation/vergence model may be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing. In some implementations, the wearable system may use ray casting techniques to determine which real or virtual objects are along the direction of the user's head pose or eye pose” in Para.[0133]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Xu in order to improve user experience with head-mounted display devices.
As per claim 4, Fukushima teaches all of limitation of claim 2.
Fukushima is silent about wherein the one or more processors and/or the circuitry configured to further execute selection processing to select the first virtual object, and wherein in the selection processing, the first virtual object is selected based on a number of times the line-of-sight of the user is directed.
Xu teaches wherein the one or more processors and/or the circuitry configured to further execute selection processing to select the first virtual object, and wherein in the selection processing, the first virtual object is selected based on a number of times the line-of-sight of the user is directed(“the user may move their head or eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes may be tracked and an accommodation/vergence model may be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing. In some implementations, the wearable system may use ray casting techniques to determine which real or virtual objects are along the direction of the user's head pose or eye pose” in Para.[0133]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Xu in order to improve user experience with head-mounted display devices.
As per claim 6, Fukushima teaches all of limitation of claim 4.
Fukushima is silent about wherein the first virtual object is a virtual object of which a number of times the line-of-sight of the user is directed reached a predetermined number of times.
Xu teaches wherein the first virtual object is a virtual object of which a number of times the line-of-sight of the user is directed reached a predetermined number of times (“the user may move their head or eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes may be tracked and an accommodation/vergence model may be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing. In some implementations, the wearable system may use ray casting techniques to determine which real or virtual objects are along the direction of the user's head pose or eye pose” in Para.[0133]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Xu in order to improve user experience with head-mounted display devices.
As per claim 8, Fukushima teaches all of limitation of claim 1.
Fukushima is silent about wherein the first virtual object is specified by a hand of the user, a finger of the user or a controller to operate the head-mounted display.
Xu teaches wherein the first virtual object is specified by a hand of the user, a finger of the user or a controller to operate the head-mounted display (“the user may move their head or eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes may be tracked and an accommodation/vergence model may be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing. In some implementations, the wearable system may use ray casting techniques to determine which real or virtual objects are along the direction of the user's head pose or eye pose” in Para.[0133], The virtual object is selected by a wearable system along the direction of the user’s head pose or eye pose.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Xu in order to improve user experience with head-mounted display devices.
Fukushima in view of Okumura
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Fukushima et al.(USPubN 2008/0218515; hereinafter Fukushima) in view of Okumura et al.(USPubN 2019/0025916; hereinafter Okumura).
As per claim 11, Fukushima teaches all of limitation of claim 1.
Fukushima is silent about wherein in the control processing, it is controlled such that weighting is performed so that intensity of the tactile sensation with the second virtual object is stronger than intensity of tactile sensation with other virtual objects, and the tactile sensation with the second virtual object is reproduced thereby.
Okumura teaches wherein in the control processing, it is controlled such that weighting is performed so that intensity of the tactile sensation with the second virtual object is stronger than intensity of tactile sensation with other virtual objects, and the tactile sensation with the second virtual object is reproduced thereby(“the character control section 43c may present the user with a tactile sense using the haptic device 253 not only when the user character C performs an action of seizing an object in the virtual space but also in the case where the user performs an action of touching an object in the virtual space. At this time, the detail, intensity, orientation, and so on of a tactile sense presented to the user may be varied in accordance with the material and weight of the object with which the user character C comes into contact and the positional relationship between the user character C and the object. This allows the user to feel a sensation as if he or she were directly touching a non-existent object in the virtual space. Also, it is possible to prevent the user from erroneously moving his or her hand to the position in the real space where the hand interferes with an object in the virtual space. According to such control, in the case where the user character C performs, for example, an action of placing its hand on a desk and removing an item on the desk in the virtual space, it is possible to convey a sensation of the desk and the item on the desk to the user although the user actually moves within a space in which there is nothing” in Para.[0069], The virtual object can have heavier weight than other virtual object so the tactile sense can be stronger on the virtual object.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Okumura in order to improve user experience with head-mounted display devices.
Fukushima in view of Dean
Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Fukushima et al.(USPubN 2008/0218515; hereinafter Fukushima) in view of Dean et al.(USPubN 2021/0216132; hereinafter Dean).
As per claim 12, Fukushima teaches all of limitation of claim 1.
Fukushima is silent about wherein in the control processing, it is controlled such that weighting is performed so that intensity of the tactile sensation with the second virtual object is stronger than intensity of tactile sensation with other virtual objects, and the tactile sensation with the second virtual object is reproduced thereby.
Dean teaches wherein in the control processing, it is controlled such that weighting is performed so that intensity of the tactile sensation with the second virtual object is stronger than intensity of tactile sensation with other virtual objects, and the tactile sensation with the second virtual object is reproduced thereby(“one or more sensory attributes associated with the virtual object can be identified, and the sensory attributes can be simulated using one or more sensory rendering devices. A sensory device can include any device that is capable of producing a sensory input (e.g., heat, cold, air current, sound, vibration, mist, etc.) detectable by human senses. The sensory input can be dynamically generated and directed to match the dynamic movement, intensity, and/or manifestation of a sensory attribute exhibited by a virtual object located in a virtual reality environment. For example, dynamic movement associated with a virtual object and/or a virtual user in relation to the virtual object can be simulated by dynamically directing sensory input generated by one or more sensory rendering devices at a location within a physical system environment (e.g., via an actuator, a track system, a cable or wire system, and/or a series of sensory rendering devices). A virtual intensity of the virtual object can be simulated by dynamically varying an amount of sensory input generated by the sensory rendering devices (e.g., by dynamically altering the degrees of voltage and/or duration of same voltage in real time, to the sensory device to vary sensory input). An intensity of a sensory input to be generated by a sensory rendering device can be calculated based in part on (i) a virtual distance between a virtual object and a virtual user and (ii) features of the virtual object, such as size, strength, force, weight, duration, composition, and the like.” in Para.[0016], The tactile sense can be stronger on the virtual object depend on distance between virtual object and the user.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Dean in order to improve user experience with head-mounted display devices.
As per claim 13, Fukushima teaches all of limitation of claim 1.
Fukushima is silent about wherein in the control processing, it is controlled such that weighting is performed so that intensity of the tactile sensation with the second virtual object becomes stronger as a distance between the first virtual object and the second virtual object is longer, and the tactile sensation with the second virtual object is reproduced thereby.
Dean teaches wherein in the control processing, it is controlled such that weighting is performed so that intensity of the tactile sensation with the second virtual object becomes stronger as a distance between the first virtual object and the second virtual object is longer, and the tactile sensation with the second virtual object is reproduced thereby (“one or more sensory attributes associated with the virtual object can be identified, and the sensory attributes can be simulated using one or more sensory rendering devices. A sensory device can include any device that is capable of producing a sensory input (e.g., heat, cold, air current, sound, vibration, mist, etc.) detectable by human senses. The sensory input can be dynamically generated and directed to match the dynamic movement, intensity, and/or manifestation of a sensory attribute exhibited by a virtual object located in a virtual reality environment. For example, dynamic movement associated with a virtual object and/or a virtual user in relation to the virtual object can be simulated by dynamically directing sensory input generated by one or more sensory rendering devices at a location within a physical system environment (e.g., via an actuator, a track system, a cable or wire system, and/or a series of sensory rendering devices). A virtual intensity of the virtual object can be simulated by dynamically varying an amount of sensory input generated by the sensory rendering devices (e.g., by dynamically altering the degrees of voltage and/or duration of same voltage in real time, to the sensory device to vary sensory input). An intensity of a sensory input to be generated by a sensory rendering device can be calculated based in part on (i) a virtual distance between a virtual object and a virtual user and (ii) features of the virtual object, such as size, strength, force, weight, duration, composition, and the like.” in Para.[0016], The user can hold the first virtual object or controller which can be a first virtual object and the tactile sense can be stronger on the virtual object depend on distance between virtual object and the user.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Fukushima with the above teachings of Dean in order to improve user experience with head-mounted display devices.
Allowable Subject Matter
Claims 5, 7, 14 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SUNGHYOUN PARK whose telephone number is (571)270-1333. The examiner can normally be reached M - Thur 6:00 am - 4 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI Q TRAN can be reached at (571)272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SUNGHYOUN PARK/Examiner, Art Unit 2484
Prosecution Insights