DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 3, 7 – 14, and 19 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Rafii et al. (U.S. PG Pub 2002/0140633) in view of Zaliva (U.S. PG Pub 2012/0007821) in view of Levesque et al. (U.S. PG Pub 2015/0145656).
Regarding Claim 1, Rafii et al. teach a system (Figure 1, Element 10. Paragraph 30), including: a controller (Figure 2A, Element 210. Paragraph 38) configured to:
control (i) one or more illumination sources (Figure 2A, Element 180. Paragraph 36) positioned to illuminate a three-dimensional (3D) scene (Figure 2A, Element 40. Paragraph 36) and (ii) one or more imaging sensors (Figure 2A, Elements 150. Paragraph 36) positioned to view the 3D scene (Figure 2A, Element 40. Paragraph 36);
acquire imaging information of the 3D scene (Figure 2A, Element 40. Paragraph 36);
provide information to an augmented reality system (Figure 2A, Element 110. Paragraph 35) within the vehicle (Figure 1, Element 20. Paragraph 30) to display, on a display (Figures 1 and 2A, Element 50. Paragraph 30), a graphical object (Figures 2A - 5B, Elements 130A - 130I. Paragraphs 40 - 42, 49 - 51 and 65 - 69) interacting with other objects displayed by display (Figures 1 and 2A, Element 50. Paragraph 30);
determine a 3D gesture (Figure 2A, Element 280, Sub-Element Routine. Paragraph 49) performed in the 3D scene (Figure 2A, Element 40. Paragraph 36);
interpret the 3D gesture (Figure 2A, Element 280, Sub-Element Routine. Paragraph 49) as a command (Figures 3A - 3H, Elements not labeled, but are the slide, rotation, and tilt. Paragraphs 46, 50, and 66); and
communicate with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to cause a display of an augmented reality presentation including a second graphical object (Figures 2A - 5B, Elements 130A - 130I. Paragraphs 40 - 42, 49 - 51 and 65 - 69) associated with the command (Figures 3A - 3H, Elements not labeled, but are the slide, rotation, and tilt. Paragraphs 46, 50, and 66).
Rafii et al. is silent with regards to a controller configured to control (iii) a haptic projector; determine a 3D gesture based, at least in part, on (i) one or more 3D primitives of a detected 3D movement and (ii) a library of 3D gesture primitives; provide haptic feedback based on the 3D gesture.
Zaliva teach determine a 3D (Paragraphs 176 and 274 – 280) gesture (Element gestures. Paragraph 288) based, at least in part, on (i) one or more 3D (Paragraphs 176 and 274 – 280) primitives (Paragraph 289) of a detected 3D movement (Paragraphs 176 and 274 – 280) and (ii) a library (Figures 41a and 41b, Element Classifier. Paragraphs 281 – 282) of 3D (Paragraphs 176 and 274 – 280) gesture primitives (Paragraph 289).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al. with the gesture primitives of Zaliva . The motivation to modify the teachings of Rafii et al. with the teachings of Zaliva is to provide for classification recognition of gesture primitives in a touch-based user interface, as taught by Zaliva (Paragraphs 10 – 17).
Levesque et al. teach a controller (Figure 1, Element 132. Paragraph 29) configured to control (iii) a haptic projector (Figure 1, Element 136. Paragraph 30); provide haptic feedback (Figure 1, Element 136. Paragraph 30) based on the 3D (Paragraph 31) gesture (Paragraph 60).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al. and the gesture primitives of Zaliva with the haptic feedback of Levesque et al. The motivation to modify the teachings of Rafii et al. and Zaliva with the teachings of Levesque et al. is to provide indirect haptic feedback, as taught by Levesque et al. (Paragraph 4).
Regarding Claim 2, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein the controller (Figure 2A, Element 210. Paragraph 38) is configured to acquire imaging information for control objects (Figures 3A - 3F, Elements 300 and 310. Paragraph 45) within view of the one or more imaging sensors (Figure 2A, Elements 150. Paragraph 36), and the imaging information for control objects (Figures 3A - 3F, Elements 300 and 310. Paragraph 45) is used to determine 3D gesture (Figure 2A, Element 280, Sub-Element Routine. Paragraph 49) information indicating a command (Figures 3A - 3H, Elements not labeled, but are the slide, rotation, and tilt. Paragraphs 46, 50, and 66) to a machine under control.
Regarding Claim 3, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein the controller (Figure 2A, Element 210. Paragraph 38) provides storage (Figure 2A, Element 280. Paragraph 38) for the library of 3D gesture (Figure 2A, Element 280, Sub-Element Routine. Paragraph 49).
Rafii et al. is silent with regards to storage for the library of 3D gesture primitives as electronic records in a database.
Zaliva teach storage for the library of 3D (Paragraphs 176 and 274 – 280) gesture primitives (Paragraph 289) as electronic records in a database (Figures 41a and 41b, Element Classifier. Paragraphs 281 – 282).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al. with the gesture primitives of Zaliva . The motivation to modify the teachings of Rafii et al. with the teachings of Zaliva is to provide for classification recognition of gesture primitives in a touch-based user interface, as taught by Zaliva (Paragraphs 10 – 17).
Regarding Claim 7, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach further including: a port (Figure 2A, Element not labeled, but is the connection between the array (Element 140) and the control processor (Element 220). Paragraphs 36 – 38) configured to receive information from a photodetector (Figure 2A, Elements 150. Paragraph 36) configured to sense a change in light intensity indicating at least one of a presence or an absence of (Paragraph 33) an object (Element hand. Paragraph 33) within the 3D scene (Figure 2A, Element 40. Paragraph 36).
Regarding Claim 8, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein the controller (Figure 2A, Element 210. Paragraph 38) is configured to interact with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to present a presentation interface displaying an augmented reality presentation on the display (Figures 1 and 2A, Element 50. Paragraph 30), wherein the presentation interface is produced based, at least in part (Figures 3G and 3H, Element 130G. Paragraph 65), on an image captured by the one or more imaging sensors (Figure 2A, Elements 150. Paragraph 36).
Regarding Claim 9, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein the controller (Figure 2A, Element 210. Paragraph 38) is configured to interact with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to present a presentation interface displaying an augmented reality presentation that includes a sequence of images of a scene (Figures 3A – 3C and 3E – 3F. Paragraphs 40 – 42 and 49 – 51) with computer generated information.
Regarding Claim 10, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein the controller (Figure 2A, Element 210. Paragraph 38) is configured to:
interact with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to present a presentation interface displaying an augmented reality presentation that includes a sequence of images of a scene (Figures 3A – 3C and 3E – 3F. Paragraphs 40 – 42 and 49 – 51) with computer generated information;
determine destination information (Element Coordinates. Paragraph 33) from at least one image of the sequence of images (Figures 3A – 3C and 3E – 3F. Paragraphs 40 – 42 and 49 – 51) and information entered or used by an operator;
select an icon (Figure 3D, Element icon. Paragraph 42) representing an advertisement relevant to the destination information (Element Coordinates. Paragraph 33); and
interact with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to display the icon (Figure 3D, Element icon. Paragraph 42) representing the advertisement in the augmented reality presentation by the display (Figures 1 and 2A, Element 50. Paragraph 30).
Regarding Claim 11, Rafii et al. teach a method comprising:
illuminating a 3D scene (Figure 2A, Element 40. Paragraph 36) comprising a portion of a vehicle (Figure 1, Element 20. Paragraph 30), the vehicle (Figure 1, Element 20. Paragraph 30) being provided with a controller (Figure 2A, Element 210. Paragraph 38) configured to control (i) one or more illumination sources (Figure 2A, Element 180. Paragraph 36) positioned to illuminate the 3D scene (Figure 2A, Element 40. Paragraph 36), and (ii) imaging sensors (Figure 2A, Elements 150. Paragraph 36) positioned to view the 3D scene (Figure 2A, Element 40. Paragraph 36);
acquiring imaging information of the 3D scene (Figure 2A, Element 40. Paragraph 36);
providing information to an augmented reality system (Figure 2A, Element 110. Paragraph 35), to display, on a display (Figures 1 and 2A, Element 50. Paragraph 30), a graphical object (Figures 2A - 5B, Elements 130A - 130I. Paragraphs 40 - 42, 49 - 51 and 65 - 69) interacting with other objects displayed by display (Figures 1 and 2A, Element 50. Paragraph 30);
determining a 3D gesture (Figure 2A, Element 280, Sub-Element Routine. Paragraph 49) performed in the 3D scene (Figure 2A, Element 40. Paragraph 36); and
communicating with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to cause a display of an augmented reality presentation including a second graphical object (Figures 2A - 5B, Elements 130A - 130I. Paragraphs 40 - 42, 49 - 51 and 65 - 69) associated with the command (Figures 3A - 3H, Elements not labeled, but are the slide, rotation, and tilt. Paragraphs 46, 50, and 66).
Rafii et al. is silent with regards to (iii) a haptic projector; determine a 3D gesture based, at least in part, on (i) one or more 3D primitives of a detected 3D movement and (ii) a library of 3D gesture primitives; and provide haptic feedback based on the 3D gesture.
Zaliva teach determine a 3D (Paragraphs 176 and 274 – 280) gesture (Element gestures. Paragraph 288) based, at least in part, on (i) one or more 3D (Paragraphs 176 and 274 – 280) primitives (Paragraph 289) of a detected 3D movement (Paragraphs 176 and 274 – 280) and (ii) a library (Figures 41a and 41b, Element Classifier. Paragraphs 281 – 282) of 3D (Paragraphs 176 and 274 – 280) gesture primitives (Paragraph 289).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al. with the gesture primitives of Zaliva . The motivation to modify the teachings of Rafii et al. with the teachings of Zaliva is to provide for classification recognition of gesture primitives in a touch-based user interface, as taught by Zaliva (Paragraphs 10 – 17).
Levesque et al. teach (iii) a haptic projector (Figure 1, Element 136. Paragraph 30); provide haptic feedback (Figure 1, Element 136. Paragraph 30) based on the 3D (Paragraph 31) gesture (Paragraph 60).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al. and the gesture primitives of Zaliva with the haptic feedback of Levesque et al. The motivation to modify the teachings of Rafii et al. and Zaliva with the teachings of Levesque et al. is to provide indirect haptic feedback, as taught by Levesque et al. (Paragraph 4).
Regarding Claim 12, Rafii et al. in view of Zaliva in view of Levesque et al. teach the method of claim 11 (See Above), comprising: acquiring imaging information for control objects (Figures 3A - 3F, Elements 300 and 310. Paragraph 45) within view of the one or more imaging sensors (Figure 2A, Elements 150. Paragraph 36); and
using the imaging information for control objects (Figures 3A - 3F, Elements 300 and 310. Paragraph 45) to determine 3D gesture (Figure 2A, Element 280, Sub-Element Routine. Paragraph 49) information indicating a command (Figures 3A - 3H, Elements not labeled, but are the slide, rotation, and tilt. Paragraphs 46, 50, and 66) to a machine under control.
Regarding Claim 13, Rafii et al. in view of Zaliva in view of Levesque et al. teach the method of claim 11 (See Above). Rafii et al. teach comprising: receiving information from a photodetector (Figure 2A, Elements 150. Paragraph 36) configured to sense a change in light intensity indicating at least one of a presence or an absence of (Paragraph 33) an object (Element hand. Paragraph 33) within the 3D scene (Figure 2A, Element 40. Paragraph 36).
Regarding Claim 14, Rafii et al. in view of Zaliva in view of Levesque et al. teach the method of claim 11 (See Above). Rafii et al. teach comprising:
determining destination information (Element Coordinates. Paragraph 33) from the imaging information (Figures 3A – 3C and 3E – 3F. Paragraphs 40 – 42 and 49 – 51) and information entered or used by an operator;
selecting an icon (Figure 3D, Element icon. Paragraph 42) representing an advertisement relevant to the destination information (Element Coordinates. Paragraph 33); and
interacting with the augmented reality system (Figure 2A, Element 110. Paragraph 35) to display the icon (Figure 3D, Element icon. Paragraph 42) representing the advertisement in the augmented reality presentation by the display (Figures 1 and 2A, Element 50. Paragraph 30).
Regarding Claim 19, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein at least one of the one or more illumination sources (Figure 2A, Element 180. Paragraph 36) is a laser (Paragraph 36).
Regarding Claim 20, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach comprising optics associated with one or more of the illumination sources (Figure 2A, Element 180. Paragraph 36) selected from the group consisting of: focusing optics, a lens (Figure 2A, Element 190. Paragraph 36), and a diffuser.
The instant Specification discloses “Some of the illumination sources 108, 110 can have associated focusing optics. In this example, six LEDs 108 (four of which are arranged at the center and two of which flank the board 172 at the sides) have focusing lenses, and ten additional LEDs 110 (which are arranged in columns of two, three, three, and two LEDs, respectively) are without focusing lenses (Paragraph 17. Emphasis Added).” The instant Specification further discloses “It should be stressed that the arrangement shown in FIG. 2 is representative and not limiting. For example, lasers or other light sources can be used instead of LEDs…In implementations that include laser(s), additional optics (e.g., a lens or diffuser) may be employed to widen the laser beam (and make its field of view similar to that of the cameras) (Paragraph 26. Emphasis Added).” Given the fact that the instant Specification disclose that the focusing lens (or focusing optics) are used with LEDs and an alternative to LEDs would be a laser that has additional elements of a lens or diffuser, it is clear that the applicant has intended to have the claim treated in the disjunctive “or” because the fact that all of the optics are used in an alternative form.
The examiner notes that the instant claim language claims the limitation “comprising optics associated with one or more of the illumination sources selected from the group consisting of: focusing optics, a lens, and a diffuser” in the conjunctive form. The Superguide Corp. v. DirecTV Enterprises, Inc., 69 USPQ2d 1865 (Fed. Cir. 2004) decision regarding the claim interpretation of “at least one of x, y, and z” on Pages 15-16 set forth the rationale for determining that plain meaning of the term “and” is conjunctive (i.e. at least one of x, at least one of y, and at least one of z). However, when the plain meaning is inconsistent with the specification and the specification shows that the meaning “and “ is in fact meant to connote a disjunctive “or”, then the “and” should treated as a disjunctive “or” for purposes of the BRI of the claim in light of the specification (i.e. “at least one of x, or at least one of y, or at least one of z”). Given the instant disclosure of Paragraphs 17 and 26 of the instant Specification it has been made clear that the applicant is attempting to have the claim limitation interpreted using the disjunctive “or” interpretation. The claim language will be treated as such.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Rafii et al. (U.S. PG Pub 2002/0140633) in view of Zaliva (U.S. PG Pub 2012/0007821) in view of Levesque et al. (U.S. PG Pub 2015/0145656) in view of Holz (U.S. PG Pub 2014/0028861).
Regarding Claim 4, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above), wherein the controller is configured to: separate information received from pixels sensitive to infrared (IR) light from information received from pixels sensitive to visible light; recognize the 3D gesture based, at least in part, on image information from one or more IR sensors; and provide image information from one or more red-green-blue (RGB) sensors as a video feed via a presentation interface.
Holz teaches wherein the controller is configured to:
separate information (Paragraph 60) received from pixels sensitive to infrared (IR) light (Paragraph 60. Element IR light) from information received from pixels sensitive to visible light (Paragraph 60. Element RGB light);
recognize the 3D gesture (Paragraph 71. Element 3D motion of object), based at least in part on image information from one or more IR sensors (Paragraph 60. Element IR channel); and
provide image information from one or more red-green-blue (RGB) sensors (Paragraph 60. Element RGB channel) as a video feed via a presentation interface.
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al., the gesture primitives of Zaliva, and the haptic feedback of Levesque et al. with the RGB-IR filter of Holz. The motivation to modify the teachings of Rafii et al., Zaliva, and Levesque et al. with the teachings of Holz is to enable improvements in receiving light, commands, communication and/or other interfacing, as taught by Holz (Paragraph 5).
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Rafii et al. (U.S. PG Pub 2002/0140633) in view of Zaliva (U.S. PG Pub 2012/0007821) in view of Levesque et al. (U.S. PG Pub 2015/0145656) in view of Ludwig et al. (U.S. PG Pub 2013/0141375).
Regarding Claim 17, Rafii et al. in view of Zaliva in view of Levesque et al. teach the system (Figure 1, Element 10. Paragraph 30) of claim 1 (See Above). Rafii et al. teach wherein the 3D gesture primitives are stored as vectors with mathematically specified spatial trajectories.
Ludwig et al. teach the 3D (Figure 42a and 42b. Paragraph 211) gesture primitives (Paragraph 156) are stored as vectors (Figure 36B, Element 140. Paragraph 184) with mathematically specified spatial trajectories (Paragraphs 175 – 181).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al., the gesture primitives of Zaliva, and the haptic feedback of Levesque et al. with the vectors of Ludwig et al. The motivation to modify the teachings of Rafii et al., Zaliva, and Levesque et al. with the teachings of Ludwig et al. is to provide for detected gesture prosody, as taught by Ludwig et al. (Paragraphs 176 – 178).
Regarding Claim 18, Rafii et al. in view of Zaliva in view of Levesque et al. teach the method of claim 11 (See Above). Rafii et al. teach wherein the 3D gesture primitives are stored as vectors with mathematically specified spatial trajectories.
Ludwig et al. teach the 3D (Figure 42a and 42b. Paragraph 211) gesture primitives (Paragraph 156) are stored as vectors (Figure 36B, Element 140. Paragraph 184) with mathematically specified spatial trajectories (Paragraphs 175 – 181).
It would have been obvious to a person of ordinary skill in the art to modify the vehicle input device of Rafii et al., the gesture primitives of Zaliva, and the haptic feedback of Levesque et al. with the vectors of Ludwig et al. The motivation to modify the teachings of Rafii et al., Zaliva, and Levesque et al. with the teachings of Ludwig et al. is to provide for detected gesture prosody, as taught by Ludwig et al. (Paragraphs 176 – 178).
Response to Arguments
All arguments are considered moot in light of the new grounds of rejection presented above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW B SCHNIREL whose telephone number is (571)270-7690. The examiner can normally be reached Monday - Friday, 10 - 6 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.B.S/Examiner, Art Unit 2625
/WILLIAM BODDIE/Supervisory Patent Examiner, Art Unit 2625
Prosecution Insights