DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 14, 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 6, 9, 14 of U.S. Patent No. 11,403,825. Although the claims at issue are not identical, they are not patentably distinct from each other because the notion of the claims does refer to the same invention and claim 1 of the current application corresponds with claims 1 and 6 of U.S. Patent 11,403,825. Claims 1 and 6 of U.S. Patent 11,403,825 anticipates claim 1 of the current application because it includes all of the limitations of the current application claim 1.
Claim 20 of the current application are merely a different statutory category to claims 9 and 14 of U.S. Patent No. 11,403,825. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the statutory category of claims 9 and 14 of U.S. Patent No. 11,403,825 from a system to a non-transitory computer readable medium as it would achieve predictable results.
Below is an example limitation mapping between the current application claim 1 and U.S. Patent No. 11,403,825 claims 1 and 6
Current Application
U.S. Patent No. 11,403,825
1. A method comprising: determining, in a mixed reality environment, a collision between a first object and a second object, wherein one or more of the first object and the second object comprises a virtual object; determining an attribute associated with the collision; determining, based on the attribute, an audio signal corresponding to the collision,
1. A method comprising: identifying a first real object in a mixed reality environment, the mixed reality environment having a user; identifying a second real object in the mixed reality environment; generating, in the mixed reality environment, a first virtual object corresponding to the second real object; identifying, in the mixed reality environment, a collision between the first real object and the first virtual object; determining a first attribute associated with the collision; determining, based on the first attribute, a first audio signal corresponding to the collision; and presenting to the user, via a first speaker, the first audio signal, wherein: the first real object comprises a sensor, and identifying the collision comprises predicting the collision based on an output of the sensor.
wherein the determining the audio signal comprises: determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.
6. The method of claim 1, wherein determining the first audio signal comprises: generating a MIDI parameter based on the first attribute; and determining, using a sound engine, the first audio signal based on the MIDI parameter.
Below is a claim mapping between the current application and U.S. Patent No. 11,403,825.
Current Application
1
14
20
U.S. Patent No. 11,403,825
1, 6
9, 14
9, 14
Claims 1, 14, 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 6, 9, 14 of U.S. Patent No. 11,657,585. Although the claims at issue are not identical, they are not patentably distinct from each other because the notion of the claims does refer to the same invention and claim 1 of the current application corresponds with claims 1 and 6 of U.S. Patent 11,657,585. Claims 1 and 6 of U.S. Patent 11,657,585 anticipates claim 1 of the current application because it includes all of the limitations of the current application claim 1.
Claim 20 of the current application are merely a different statutory category to claims 9 and 14 of U.S. Patent No. 11,657,585. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the statutory category of claims 9 and 14 of U.S. Patent No. 11,657,585 from a system to a non-transitory computer readable medium as it would achieve predictable results.
Below is a limitation mapping between the current application claim 1 and U.S. Patent 11,657,585 claim 1 and 6
Current Application
U.S. Patent 11,657,585
1. A method comprising: determining, in a mixed reality environment, a collision between a first object and a second object, wherein one or more of the first object and the second object comprises a virtual object; determining an attribute associated with the collision; determining, based on the attribute, an audio signal corresponding to the collision,
1. A method comprising: identifying a first real object in a mixed reality environment; identifying a first virtual object in the mixed reality environment, wherein a position of the first virtual object in the mixed reality environment is based on a position of the first real object in the mixed reality environment; identifying a second real object in the mixed reality environment; identifying, in the mixed reality environment, a collision between the first virtual object and a second virtual object corresponding to the second real object; determining a first attribute associated with the collision; determining, based on the first attribute, a first audio signal corresponding to the collision; and presenting to a user, via a first speaker, the first audio signal, wherein: the first real object comprises a sensor, and identifying the collision comprises predicting the collision based on an output of the sensor.
wherein the determining the audio signal comprises: determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.
6. The method of claim 1, wherein determining the first audio signal comprises: generating a MIDI parameter based on the first attribute; and determining, using a sound engine, the first audio signal based on the MIDI parameter.
Below is a claim mapping between the current application and U.S. Patent 11,657,585
Current Application
1
14
20
U.S. Patent 11,657,585
1, 6
9, 14
9, 14
Claims 1-9, 14-15, 18, 19, 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-8, 12, 20 of U.S. Patent No. 11,978,171. Although the claims at issue are not identical, they are not patentably distinct from each other because the notion of the claims does refer to the same invention and claim 1 of the current application corresponds with claims 1 of U.S. Patent 11,978,171. Claims 1 of U.S. Patent 11,978,171 anticipates claim 1 of the current application because it includes all of the limitations of the current application claim 1.
Claims 11, 12, 13 and 16, 17 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 9 of U.S. Patent No. 11,978,171. in view of McCulloch et al. (US 2013/0286004)(Hereinafter referred to as McCulloch)
U.S. Patent No. 11,978,171. teaches all of the limitations of Claims 11, 12, 13, 16 and 17 except the limitations “wherein one or more of the first object and the second object is identified based on an output of one or more sensors” “wherein a wearable head device comprises one or more sensors” “wherein the one or more sensors comprise a camera, a microphone, an inertial measurement unit, a LIDAR sensor, a GPS sensor, a magnetic sensor, or any combination thereof.”.
McCulloch teaches a virtual interaction between two real objects table and floor in a virtual manner, virtual glass falling on or colliding with the floor and utilizing an outward facing camera for object recognition to detect and track the objects (The outward facing cameras 113 in conjunction with the object recognition engine 192 and gesture recognition engine 193 implements a natural user interface (NUI) in embodiments of the display device system 8. See paragraph [0099])( The object recognition engine 192 of the image and audio processing engine 191 detects, tracks and identifies real objects in the display field of view and the 3D environment of the user based on captured image data and depth data if available or determined depth positions from stereopsis. See paragraph [0086]) (For the real object, its physical properties are identified based on captured image data and object reference data in at least one network accessible datastore. See paragraph [0029]) (As shown in FIG. 3B, the one or more force vectors were concentrated in a small enough area of the glass to cause a cracking criteria to be satisfied as illustrated by various cracks like representative crack 35N and was also enough for a complete break in the table top resulting in small pieces like representative 37N falling on the floor. The cracking and breaking of the glass would take a significant amount of the energy in the collision, leaving the basketball 305 with an amount of energy under the conservation of energy principle for a very small bounce or maybe a roll off the table. Eventually, the basketball 30 rolls and stops near Joe's feet.) (As shown in FIG. 3B, the one or more force vectors were concentrated in a small enough area of the glass to cause a cracking criteria to be satisfied as illustrated by various cracks like representative crack 35N and was also enough for a complete break in the table top resulting in small pieces like representative 37N falling on the floor. The cracking and breaking of the glass would take a significant amount of the energy in the collision, leaving the basketball 305 with an amount of energy under the conservation of energy principle for a very small bounce or maybe a roll off the table. Eventually, the basketball 30 rolls and stops near Joe's feet. See paragraph [0072] )( FIG. 1A is a block diagram depicting example components of an embodiment of a see-through, augmented or mixed reality display device system. System 8 includes a see-through display device as a near-eye, augmented reality display device 2 in communication with a processing unit 4 via a wire 6 in this example or wirelessly in other examples. In this embodiment, head mounted, display device 2 is in the shape of eyeglasses in a frame 115, with a display optical system 14 for each eye in which image data is projected into a user's eye to generate a display of the image data while a user also sees through the display optical systems 14 for an actual direct view of the real world. Each display optical system 14 is also referred to as a see-through display, and the two display optical systems 14 together may also be referred to as a see-through display. See paragraph [0035])(collision between glass and the floor).
U.S. Patent 11,978,171 and McCulloch teach interactions between virtual objects and representing real objects as virtual objects and McCulloch teaches that physics based properties can be represented for virtual objects created from physical objects and tracked using an outward facing camera mounted on an HMD, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of U.S. Patent 11,978,171 with the relative positioning and detection of the virtual objects to the real objects and the physical interaction of virtual objects representative of real objects in the real environment such that the user would have a more realistic interaction with the virtual representation.
Below is a limitation mapping between the current application claim 1 and U.S. Patent 11,978,171 claim 1
Current Application
U.S. Patent 11,978,171
1. A method comprising: determining, in a mixed reality environment, a collision between a first object and a second object, wherein one or more of the first object and the second object comprises a virtual object; determining an attribute associated with the collision; determining, based on the attribute, an audio signal corresponding to the collision, wherein the determining the audio signal comprises: determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.
1. A method comprising: identifying a first object in a mixed reality environment; identifying a second object in the mixed reality environment, wherein one or more of the first object and the second object comprises a virtual object; determining, via a sensor of a wearable head device, a position of the second object in a coordinate space associated with a user; determining, in the mixed reality environment, a collision between the first object and the second object, wherein the collision is determined based on the determined position of the second object; determining an attribute associated with the collision; determining, based on the attribute, an audio signal corresponding to the collision, wherein the determining the audio signal comprises: generating a MIDI parameter based on the attribute; and determining, via a sound engine, the audio signal based on the MIDI parameter; and presenting to the user, via a speaker, the audio signal.
Below is a claim mapping between the current application and U.S. Patent 11,978,171
Current Application
1
2
3
4
5
6
7
8, 1
9, 1
14
15
18, 14
19, 14
20
U.S. Patent 11,978,171
1
2
3
4
5
6
7
1
1
8
12
8
8
20
Claims 1-10, 14-15, 18, 19, 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-7, 9-13, 16, 17, 20 of U.S. Patent No. 12,254,582. Although the claims at issue are not identical, they are not patentably distinct from each other because the notion of the claims does refer to the same invention and claim 1 of the current application corresponds with claims 1 of U.S. Patent 12,254,582. Claims 1 of U.S. Patent 12,254,582 anticipates claim 1 of the current application because it includes all of the limitations of the current application claim 1.
Claims 11, 12, 13 and 16, 17 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 9 of U.S. Patent No. 12,254,582. in view of McCulloch et al. (US 2013/0286004)(Hereinafter referred to as McCulloch)
U.S. Patent No. 12,254,582. teaches all of the limitations of Claims 11, 12, 13, 16 and 17 except the limitations “wherein one or more of the first object and the second object is identified based on an output of one or more sensors” “wherein a wearable head device comprises one or more sensors” “wherein the one or more sensors comprise a camera, a microphone, an inertial measurement unit, a LIDAR sensor, a GPS sensor, a magnetic sensor, or any combination thereof.”.
McCulloch teaches a virtual interaction between two real objects table and floor in a virtual manner, virtual glass falling on or colliding with the floor and utilizing an outward facing camera for object recognition to detect and track the objects (The outward facing cameras 113 in conjunction with the object recognition engine 192 and gesture recognition engine 193 implements a natural user interface (NUI) in embodiments of the display device system 8. See paragraph [0099])( The object recognition engine 192 of the image and audio processing engine 191 detects, tracks and identifies real objects in the display field of view and the 3D environment of the user based on captured image data and depth data if available or determined depth positions from stereopsis. See paragraph [0086]) (For the real object, its physical properties are identified based on captured image data and object reference data in at least one network accessible datastore. See paragraph [0029]) (As shown in FIG. 3B, the one or more force vectors were concentrated in a small enough area of the glass to cause a cracking criteria to be satisfied as illustrated by various cracks like representative crack 35N and was also enough for a complete break in the table top resulting in small pieces like representative 37N falling on the floor. The cracking and breaking of the glass would take a significant amount of the energy in the collision, leaving the basketball 305 with an amount of energy under the conservation of energy principle for a very small bounce or maybe a roll off the table. Eventually, the basketball 30 rolls and stops near Joe's feet.) (As shown in FIG. 3B, the one or more force vectors were concentrated in a small enough area of the glass to cause a cracking criteria to be satisfied as illustrated by various cracks like representative crack 35N and was also enough for a complete break in the table top resulting in small pieces like representative 37N falling on the floor. The cracking and breaking of the glass would take a significant amount of the energy in the collision, leaving the basketball 305 with an amount of energy under the conservation of energy principle for a very small bounce or maybe a roll off the table. Eventually, the basketball 30 rolls and stops near Joe's feet. See paragraph [0072] )( FIG. 1A is a block diagram depicting example components of an embodiment of a see-through, augmented or mixed reality display device system. System 8 includes a see-through display device as a near-eye, augmented reality display device 2 in communication with a processing unit 4 via a wire 6 in this example or wirelessly in other examples. In this embodiment, head mounted, display device 2 is in the shape of eyeglasses in a frame 115, with a display optical system 14 for each eye in which image data is projected into a user's eye to generate a display of the image data while a user also sees through the display optical systems 14 for an actual direct view of the real world. Each display optical system 14 is also referred to as a see-through display, and the two display optical systems 14 together may also be referred to as a see-through display. See paragraph [0035])(collision between glass and the floor).
U.S. Patent 12,254,582 and McCulloch teach interactions between virtual objects and representing real objects as virtual objects and McCulloch teaches that physics based properties can be represented for virtual objects created from physical objects and tracked using an outward facing camera mounted on an HMD, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of U.S. Patent 12,254,582 with the relative positioning and detection of the virtual objects to the real objects and the physical interaction of virtual objects representative of real objects in the real environment such that the user would have a more realistic interaction with the virtual representation.
Below is a limitation mapping between the current application claim 1 and U.S. Patent 12,254,582 claim 1
Current Application
U.S. Patent 12,254,582
1. A method comprising: determining, in a mixed reality environment, a collision between a first object and a second object, wherein one or more of the first object and the second object comprises a virtual object; determining an attribute associated with the collision; determining, based on the attribute, an audio signal corresponding to the collision, wherein the determining the audio signal comprises: determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.
1. A method comprising: wherein one or more of the first object and the second object comprises a virtual object; determining, in the mixed reality environment, a collision between the first object and the second object; determining an attribute associated with the collision; determining, based on the attribute, an audio signal corresponding to the collision, wherein the determining the audio signal comprises: generating a MIDI parameter based on the attribute; and determining, via a sound engine, the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.
identifying a first object in a mixed reality environment; identifying a second object in the mixed reality environment,
Below is a claim mapping between the current application and U.S. Patent 12,254,582
Current Application
1
2
3
4
5
6
7
8
9
10
14
15
18
19
20
U.S. Patent 12,254,582
1
2
3
4
5
6
7
9
10
11
12
13
16
17
20
Allowable Subject Matter
Claims 1-20 would be allowable if rewritten or amended to overcome the double patenting rejections, or by filing a terminal disclaimer overcoming the rejection set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter: The closest prior art of record is McCulloch et al. (US 2013/0286004)(Hereinafter referred to as McCulloch).
McCulloch teaches a virtual interaction between two real objects table and floor in a virtual manner, virtual glass falling on or colliding with the floor and utilizing an outward facing camera for object recognition to detect and track the objects (The outward facing cameras 113 in conjunction with the object recognition engine 192 and gesture recognition engine 193 implements a natural user interface (NUI) in embodiments of the display device system 8. See paragraph [0099])( The object recognition engine 192 of the image and audio processing engine 191 detects, tracks and identifies real objects in the display field of view and the 3D environment of the user based on captured image data and depth data if available or determined depth positions from stereopsis. See paragraph [0086]) (For the real object, its physical properties are identified based on captured image data and object reference data in at least one network accessible datastore. See paragraph [0029]) (As shown in FIG. 3B, the one or more force vectors were concentrated in a small enough area of the glass to cause a cracking criteria to be satisfied as illustrated by various cracks like representative crack 35N and was also enough for a complete break in the table top resulting in small pieces like representative 37N falling on the floor. The cracking and breaking of the glass would take a significant amount of the energy in the collision, leaving the basketball 305 with an amount of energy under the conservation of energy principle for a very small bounce or maybe a roll off the table. Eventually, the basketball 30 rolls and stops near Joe's feet.) (As shown in FIG. 3B, the one or more force vectors were concentrated in a small enough area of the glass to cause a cracking criteria to be satisfied as illustrated by various cracks like representative crack 35N and was also enough for a complete break in the table top resulting in small pieces like representative 37N falling on the floor. The cracking and breaking of the glass would take a significant amount of the energy in the collision, leaving the basketball 305 with an amount of energy under the conservation of energy principle for a very small bounce or maybe a roll off the table. Eventually, the basketball 30 rolls and stops near Joe's feet. See paragraph [0072] )(collision between glass and the floor).
McCulloch is silent to the limitations “determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.” Of claim 1 when read in light of the rest of the limitations in claim 1 and thus claim 1 contains allowable subject matter.
The prior art of record alone or in combination is silent to the limitations “determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.” Of claim 14 when read in light of the rest of the limitations in claim 14 and thus claim 14 contains allowable subject matter.
The prior art of record alone or in combination is silent to the limitations “determining a MIDI parameter based on the attribute; and determining the audio signal based on the MIDI parameter; and presenting, via a speaker, the audio signal.” Of claim 20 when read in light of the rest of the limitations in claim 20 and thus claim 20 contains allowable subject matter.
Claims 2-13 and 15-19 contain allowable subject matter because they depend on a claim that contains allowable subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Shotton et al. (US 2017/0185141)(Hereinafter referred to as Shotton), generally teaches interacting with an augmented reality virtual keyboard (See paragraph [0101]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS R WILSON whose telephone number is (571)272-0936. The examiner can normally be reached M-F 7:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (572)-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS R WILSON/Primary Examiner, Art Unit 2611
Prosecution Insights