DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1-9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed 2/25/2026 have been fully considered but they are not persuasive. Applicant argues, on pages 2-3 of the Response, that Jenkins teaches drawing a line within an augmented reality environment wherein the user is forced to interact and point within the field of view of the screen of the device. Examiner notes that this is exactly the concept shown in the Figure 1 of the instant application. Examiner also notes that the claims only recite that “the pointing gesture is aligned with the object relative to the field of vision of the user and not the imaging device;” (emphasis added). It should be noted that there is no language in claim 10 suggesting that the gesture has to be performed in a user’s field of view (FOV) or an imaging devices FOV. The language in claim 10 merely says that the pointing gesture is “aligned with” and “relative [to]” the user’s FOV. Examiner notes that a user can only point at what they want to select based on their own point of view (which is the only way in which a user could be sure what they were pointing at). Examiner also notes that augmented reality comprises a mixture of real-world elements and virtual elements (see Jenkins paragraphs 0011 and 0049).
Applicant further argues, on page 2 of the Response, that although Jenkins discloses a line cast from the user’s finger but does not a three dimensional line. Examiner notes that Jenkins does use a 3D modeling-based algorithms (see paragraph 0046). Examiner also notes that the line would inherently have a starting and ending X/Y/Z coordinate and thus must constitute a three-dimensional line.
Applicant also argues, on page 2 of the Response, that Jenkins discloses that the line is extended from the user’s finger and is not responsive to a pointing gesture. Examiner respectfully disagrees. Jenkins most certainly discloses that a pointing gesture is used to select an object (see Jenkins paragraph 0015, 0019, and 0052 – specifically note the mentions of the user performing a pointing gesture).
Claim Objections
Claims 1 and 10 are objected to because of the following informalities:
Claim 1 recites the language “…the isolated object.” in the last line of the claim. Examiner recommends rephrasing the above language to read --…only the object.—in order to place the claim in better form and obviate any antecedent basis issues.
Claim 10 recites the phrase “…associating the line with…” in the last line of the claim. Examiner suggests amending the above phrase to read --…associating the three-dimensional vector with…-- in order to place the claim in better form and obviate any antecedent basis issues.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Iyer et al. (hereinafter Iyer – US Doc. No. 20160091964) in view of Jenkins et al. (hereinafter Jenkins – US Doc. No. 20240096028).
Examiner notes that some of the rejections below are similar to the rejections as presented in the previous Non-Final Office Action but may include amended claim language and/or additional citations or explanation for clarity.
Regarding claim 1, Iyer discloses a system for interacting with a mobile device comprising: a mobile device (see paragraph 0015) having an imaging device (Figure 1, element 102) adapted to obtain image data of a scene of the real world in the vicinity of a user (see paragraph 0020; see also paragraph 0002 – note that this device is an AR device and AR contains real-world scenery), wherein the imaging device is positioned to simultaneously capture image data related to an object in the scene and a user's appendage positioned within a field-of-view of the imaging device (as shown in Figure 3); a computing module that receives the image data from the imaging device (Figure 11, element 1102), wherein the computing module is adapted to: identify a pointing gesture of the appendage( as detailed in paragraph 0054), and isolate the object from a remainder of the scene (as detailed in paragraph 0054), wherein the pointing gesture is aligned with the object relative to a field of vision of the user and not the imaging device (the user pointing at an object inherently uses alignment relative to/within the user’s field of vision otherwise the user wouldn’t be able to determine what he/she/they are pointing at); a screen for displaying the object (Figure 11, element 1110). Iyer does not disclose determining a three-dimensional vector, casting a ray from the finger to the object.
Jenkins discloses a wearable device (Figure 4, element 304) for displaying an AR environment (see paragraphs 0011 and 0049) wherein the user can select in object in the environment by pointing at it (see paragraph 0049) wherein the computing module, determines a three-dimensional vector of the appendage (see paragraphs 0020 and 0047), based on the three-dimensional vector, cast a ray from the finger to identify the object image data (see Figures 3 and 4, specifically element 402; see also paragraph 0047 and 0052).
It would have been obvious to combine the wearable AR device as disclosed by Iyer with the AR device which cases a virtual line to the object that the user is selecting as disclosed by Jenkins, the combination yielding predictable results and no more than one of ordinary skill in the art would expect from such an arrangement and further it would allow the user to select the intended object with increased reliability.
Regarding claim 2, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer does disclose an imaging device and Iyer also discloses that the gesture module 106 uses a three-dimensional recognition method to detect gestures (see paragraphs 0026). Iyer does not specifically disclose that the imaging device comprises a device capable of obtaining three-dimensional data.
It would have been obvious that if the gesture recognition module uses three-dimensional data, and the data is provided by the imaging module, then the imaging module must be able to obtain three-dimensional data, this would yield predictable results and no more than one of ordinary skill in the art would expect from such an arrangement.
Regarding claim 3, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer further discloses that the imaging device comprises at least one of a camera, wide-angle camera, stereo camera, depth camera, and lidar device (see paragraph 0017).
Regarding claim 4, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer further discloses that the computing module is contained within the mobile device (as shown in Figure 11 – note processor 1102).
Regarding claim 5, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Although Iyer discloses that multiple elements of the device can be networked (see paragraphs 0050 and 0075), Iyer does not specifically state that the computing module is remotely connected to the mobile device.
It would have been obvious to one having ordinary skill in the art to process the information on a remote processor, since it has been held that constructing a formerly integral structure in various elements involves only routine skill in the art. Nerwin v. Erlichman, 168 USPQ 177, 179.
Regarding claim 6, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer further discloses that the appendage is a finger (see Figure 4, element 426).
Regarding claim 7, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer further discloses that the mobile device comprises a mobile phone, headset, electronic glasses, pin, button, or a similar mobile electronic device (see paragraph 0015).
Regarding claim 8, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer further discloses that a microphone adapted to receive audio data (Figure 1, element 114).
Regarding claim 9, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Iyer further discloses that the audio data comprises a verbal keyword, wherein the keyword triggers a function in the computing module (see paragraphs 0029 and 0033).
Regarding claim 10, Iyer discloses a method of identifying an object contained within a field of view of an imaging device comprising: capturing image data of a scene in the real world contained within the field of view of the imaging device (Figure 9, element 902; see also paragraph 0020; see also paragraph 0002 – note that this device is an AR device and AR contains real-world scenery); transmitting the image data to a computing module (see paragraph 0020-0021); wherein the pointing gesture is aligned with the object relative to a field of vision of the user and not the imaging device (the user pointing at an object inherently uses alignment relative to/within the user’s field of vision otherwise the user wouldn’t be able to determine what he/she/they are pointing at); using the computing module, identifying an appendage of the user in the scene and, if an appendage is present, further determining whether a portion of the appendage is in a pointing gesture (see paragraphs 0052-0053). Iyer does not disclose casting a line to the object selected by the user.
Jenkins discloses a wearable device (Figure 4, element 304) for displaying an AR environment (see paragraphs 0011 and 0049) wherein the user can select in object in the environment by pointing at it (see paragraph 0049) wherein the computing module, determines a three-dimensional vector of the appendage (see paragraphs 0020 and 0047), based on the three-dimensional vector, cast a ray from the finger to identify the object image data (see Figures 3 and 4, specifically element 402; see also paragraph 0047 and 0052).
It would have been obvious to combine the wearable AR device as disclosed by Iyer with the AR device which cases a virtual line to the object that the user is selecting as disclosed by Jenkins, the combination yielding predictable results and no more than one of ordinary skill in the art would expect from such an arrangement and further it would allow the user to select the intended object with increased reliability.
Regarding claim 11, the combination of Iyer and Jenkins discloses all of the limitations of claim 10 as discussed in the claim 10 rejection above. Iyer further discloses capturing audio data using a microphone; and using the computing module, isolating an utterance from the audio data; and using the computing module, providing context about the object based on the utterance (see Figures 0029 and 0062).
Regarding claim 12, the combination of Iyer and Jenkins discloses all of the limitations of claim 10 as discussed in the claim 10 rejection above. Nether Iyler or Jenkins disclose comparing a size of the appendage to determine if the appendage is the user's appendage or another appendage contained in the scene. Iyer does disclose that the image rendering module is able to receive parameters [including size] (see paragraph 0048).
It would have been obvious to use the size parameter as disclosed by Iyer in order to differentiate a user’s appendage from another appendage, this would yield predictable results and no more than one of ordinary skill in the art would expect from such an arrangement.
Regarding claim 13, the combination of Iyer and Jenkins discloses all of the limitations of claim 10 as discussed in the claim 10 rejection above. Iyer further discloses that the gesture recognition module uses a skeletal-based method of gesture recognition [read: identifying joint angles of an index finger on the appendage] (see paragraph 0026).
Jenkins further discloses identifying whether fingers other than the index finger are tucked towards the appendage (see paragraph 0020).
It would have been obvious to combine the wearable AR device as disclosed by Iyer with the AR device which including finger curling determination as disclosed by Jenkins, the combination yielding predictable results and no more than one of ordinary skill in the art would expect from such an arrangement.
Regarding claim 14, the combination of Iyer and Jenkins discloses all of the limitations of claim 13 as discussed in the claim 13 rejection above. Iyer further discloses identifying keypoints on the index finger (see paragraph 0026).
Regarding claim 15, the combination of Iyer and Jenkins discloses all of the limitations of claim 10 as discussed in the claim 10 rejection above. Iyer further discloses that capturing image data of a scene contained within the field of view of the imaging device begins only after initiation by a verbal trigger (see paragraph 0029).
Regarding claim 16, the combination of Iyer and Jenkins discloses all of the limitations of claim 10 as discussed in the claim 10 rejection above. Iyer further discloses identifying a wake gesture in the image data (see paragraph 0044 note that any gesture could be a wake gesture).
Regarding claim 17, the combination of Iyer and Jenkins discloses all of the limitations of claim 16 as discussed in the claim 16 rejection above. Neither Iyer nor Jenkins discloses that the wake gesture comprises a finger pointing pose.
It would have been an obvious matter of design choice to make a wake gesture any gesture that the device can recognize since the applicant has not disclosed that a finger pointing pose solves any stated problem or is for any particular purpose and it appears that the invention would perform equally well with any gesture.
Regarding claim 18, the combination of Iyer and Jenkins discloses all of the limitations of claim 10 as discussed in the claim 10 rejection above. Iyer further discloses using the object as an input to an application or an AI agent (see paragraph 0057).
Regarding claim 19, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Neither Iyer nor Jenkins specifically discloses that the user’s appendage is not rendered to the user via the screen. While Iyer does disclose that the system may (emphasis added) render the user’s appendage over a still image (paragraph 0055), Iyer does not specify specifically ot rendering the hand. Jenkins discloses overlaying virtual images over the real world via the AR device (see paragraph 0020), but does not disclose rendering or displaying the user’s fingers/hands/arms/appendages.
It would have been obvious to one of ordinary skill in the art to not render the user’s appendages in an AR environment since the user can already physically see their hands through the AR display since this would also conserve processing cycles and save on battery by not rendering items which already exist in the real world.
Regarding claim 20, the combination of Iyer and Jenkins discloses all of the limitations of claim 1 as discussed in the claim 1 rejection above. Neither Iyer nor Jenkins specifically discloses that the scene captured by the imaging device is not rendered to the user via the screen. While Iyer discloses that the system may (emphasis added) provide un-augmented image data corresponding to the image captured (paragraph 0048), Iyer does not specify specifically not rendering the real world scene captured by the imaging device. Jenkins discloses overlaying virtual images over the real world via the AR device (see paragraph 0020), but does not disclose rendering or displaying real world scenes via the display.
It would have been obvious to one of ordinary skill in the art to not render real world scenes/objects in an AR environment since the user can already physically see the real world scenes/objects through the AR display since this would also conserve processing cycles and save on battery by not rendering items which already exist in the real world.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADAM R GIESY whose telephone number is (571)272-7555. The examiner can normally be reached Mon-Fri 8-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick Edouard can be reached at 5712727603. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADAM R. GIESY/Primary Examiner, Art Unit 2622
Prosecution Insights