DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission including amendment to the claims filed on 6/01/2026 has been entered.
3. Claims 1-4, 6-10, and 14-15 have been amended. Claims 1-20 are pending and will be considered for examination.
4. The double patenting rejection is not withdrawn until a terminal disclaimer is actually filed and approved.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(a):
(a) IN GENERAL--The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), first paragraph:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
5. Independent Claims 1 and 9-10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
Independent Claim 1 recites the limitation "responsive to detection of a gesture based, at least in part, on an image of the live video stream, interpreting the detected gesture as selecting a virtual item from a library of virtual interactive items". These limitations lack written description support in the Applicant's specification. Instead, it consistently refers to gesture capture via a sensor interface, motion capture, and detection in a 3D sensory space, which could encompass various sensing modalities beyond camera image analysis. Moreover, it is unclear what, if anything, corresponds to the claimed “image of the live video stream.” Paragraph [0082] describes rendering computer-generated imagery, including the virtual grid and virtual items, onto the field of view of a live camera (further see paragraphs [0023], [0074], [0078], [0125], [0142]). However, the specification does not identify whether the relevant image is (i) the live camera image itself, (ii) the augmented reality view in which virtual items are overlayed on the live camera image, or (iii) some other representation. More importantly, regardless of which image is intended, the specification does not support the limitations. Rather, it discloses only that a gesture is detected in a 3D sensory space and then interpreted as selecting a virtual item. Supported for this limitation is requested for further consideration. If the Examiner has overlooked the portion of the original Specification that describes this feature/term of the present invention, then Applicant should point it out (by page number and line number) in the response to the Office Action.
Independent Claims 9 and 10 recite similar limitations and are rejected under similar rational.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-4, 6-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over in view of Yang et al. (US 10281987 B1) in view of Kuroda, Yosuke (US 2012/0056992 A1).
As in Claim 1, Yang teaches a method including:
providing a live video stream for display via a wearable device, wherein the live video stream comprises an image of an object captured by a camera, (col. 35, line 66 to col. 36, line 13, col. 36, lines 34-48, col. 38, lines 4-7, a camera returns a digital video stream of the real world, which is generated by a live camera embedded in a wearable google or HMD for real-time rendering, and the camera may be implemented in a head-mounted display; further see col. 14, line 49 to col. 15, line 4, line 13, col. 36, lines 34-58, col. 38, lines 4-27 ), wherein the object comprises an identification marker (col. 37, line 34 to col. 38, lines 27, the digital video stream displays real world items/objects and images of real-world markers, which may be images of real-world objects/items, and each marker corresponds to one or more interactive virtual item);
responsive to identification of a virtual interactive item corresponding to the object based, at least in part, on the identification marker, generating imagery based, at least in part, on the identified virtual interactive item (col. 37, line 34 to col. 38, lines 27, the system generates computer-generated imagery corresponding to interactive virtual items retrieved from an AR library based on recognized markers);
providing the generated imagery for display via the wearable device (col. 37, line 34 to col. 38, lines 27, the system displaying the interactive virtual items through the wearable display device such as the wearable google or HMD).
Yang does not appear to explicitly teach responsive to detection of a gesture based, at least in part, on an image of the live video stream, interpreting the detected gesture as selecting a virtual item from a library of virtual interactive items.
However, in the same filed of the invention, Kuroda teaches responsive to detection of a gesture based, at least in part, on an image of the live video stream, interpreting the detected gesture as selecting a virtual item from a library of virtual interactive items (FIGS. 3-11, pars. 107-109, 111, 124-125, 128-134, 136-164, the system detects gestures on an image/marker OBM on the live-view images, the marker can be identified and corresponding virtual object OBV, which is stored in the object data store section 174, can be appeared in the live view image).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the device (HMD) for generating the live video stream including markers for the objects, as taught by Yang, and to select the virtual object based on the hand gesture on the live-view images, as taught by Kuroda. The motivation is to provide an enhanced augmented reality experience by enabling the displayed virtual object to interact naturally with the user’s hand through depth-based rendering, collision response, sound, animations and movement.
As in Claim 2, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches superimposing the identified virtual interactive item onto the identification marker (Yang, col. 37, lines 49-59, col. 38, lines 15-18, the interactive virtual item corresponding to the marker is superimposed upon the real-world image and placed into real-world image registration with its corresponding marker).
As in Claim 3, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda teaches that the wearable device includes one or more projectors that project imagery into a three-dimensional (3D) space (col. 36, lines 34-58, col. 37, line 60 - col. 38, line 3 and lines 38-48, the head-mounted AR system having projectors that project virtual imagery into a real-world environment and a marker-based association between real-world markers and interactive virtual items), and wherein the method includes:
projecting, by the one or more projectors of the wearable device, the identified virtual interactive item onto the identification marker (col. 36, lines 34-58, col. 37, line 60 - col. 38, line 3 and lines 38-48, the system can project the virtual items into the real-world image with its corresponding marker).
As in Claim 4, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches that the identifying of the virtual interactive item identifies two or more virtual interactive items, from the library of virtual interactive items, that provide information about the identification marker (Yang, col. 37, line 34 to col. 38, lines 27, the system identifies marker images in the digital video stream, retrieves one ore more corresponding virtual items from the AR library, and displays the retrieved virtual items as generated AR imagery superimposed on the real-world image ; and
the identified two or more virtual interactive items are included in the generated imagery (Yang, col. 37, line 34 to col. 38, lines 27).
As in Claim 6, Yang-AA Yang-Kuroda AA teaches the limitations of Claim 1. Yang-Kuroda further teaches that the identified virtual interactive item is superimposed in place of the identification marker (Yang, col. 37, lines 49-59, col. 38, lines 15-18, the interactive virtual item corresponding to the marker is superimposed upon the real-world image and placed into real-world image registration with its corresponding marker).
As in Claim 7, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches that the identification marker comprises at least one of a two-dimensional barcode or a three-dimensional barcode (Yang, col. 37, liens 41-51, , the markers can be two- or three-dimensional barcodes).
As in Claim 8, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches that the identification marker comprises an image (Yang, col. 37, liens 41-51, the markers can be thumbnail images of the virtual images that are matched to the markers. The marker can also be an image of a real-world item which the software has been programmed to recognize)
Claims 9 and 10 are substantially similar to Claim 1 and rejected under the same rationale.
As in Claim 11, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches executing an action associated with the selected virtual item (Kuroda, FIGS. 3-11, pars. 109, 111, 128-134, 145-164, based on the depth information of the user’s hand OBH, the system renders the virtual object with proper occlusion, performs hit determination, and execute animations, sound output, or movement of the virtual object).
Claims 12 and 13 are substantially similar to Claim 11 and rejected under the same rationale.
Claims 14 and 15 are substantially similar to Claim 2 and rejected under the same rationale.
As in Claim 16, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches comprising superimposing the generated imagery as a modality in a three-dimensional (3D) space (Yang, col. 36, lines 13-33, col. 39, lines 1-35, col. 44, liens 5-29, virtual modality combines rendered 3D virtual imagery with a view of the real world. The rendered 3D virtual imagery is superimposed as free-floating virtual modalities in real-world physical environment).
As in Claim 18, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda further teaches that wherein the wearable device is a head-mounted device (Yang, col. 26, liens 17-22, col. 28, line 67 to col. 29, line 2, wearable devices, e.g., googles, head mounted displays (HMD), wrist computers, heads up displays (HUDs)).
Claims 19 and 20 are substantially similar to Claim 18 and rejected under the same rationale.
7. Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over in view of Yang et al. (US 10281987 B1) in view of Kuroda, Yosuke (US 2012/0056992 A1) and further in view of Hilliges et al. (US 2014/0104274 Al).
As in Claim 5, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda does not teach that the detected gesture is a scooping gesture in which a representation of a hand appears to start from a position behind a virtual item and then proceed in a motion that appears to scoop up the virtual item from behind.
However, in the same field of the invention, Hilliges teaches that the detected gesture is a scooping gesture in which a representation of a hand appears to start from a position behind a virtual item and then proceed in a motion that appears to scoop up the virtual item from behind (pars. 23, 38, 44, a user may interact with virtual objects using gestures, such as scooping them up. When performing a scooping gesture, the physics simulator models the forces, allowing the virtual object to be lifted or moved by the user's hand).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the device (HMD) for generating the live video stream including markers for the objects, as taught by Yang, in view of Kuroda’s teachings, and to incorporate the gesture inputs with the virtual objects, as taught by Hilliges. The motivation is to create a more natural and intuitive interaction between the user and virtual objects, allowing the user to manipulate them with realistic gestures, such as scooping.
8. Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over in view of Yang et al. (US 10281987 B1) in view of Kuroda, Yosuke (US 2012/0056992 A1) and further in view of Poulos et al. (US 2014/0306993 A1).
As in Claim 17, Yang-Kuroda teaches the limitations of Claim 1. Yang-Kuroda does not teach that the generated imagery comprises a modality displaying a plurality of virtual items arranged in a grid.
However, in the same field of the invention, Poulos teaches that the generated imagery comprises a modality displaying a plurality of virtual items arranged in a grid (FIGS. 4-5, 53-55, 57, 69, 77, the virtual objects can be positioned relative to the virtual grid lines).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the device (HMD) for generating the live video stream including markers for the objects, as taught by Yang, in view of Kuroda’s teachings, and to present the virtual objects in a grid, as taught by Poulos. The motivation is to organize and align objects accurately in the AR space, making placement, scaling, and interaction more precisely and visually consistent.
Response to Arguments
9 Applicant's arguments with respect to the claims 1-20 have been fully considered, but are moot in view of the new ground(s) of rejection.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Rinna Yi whose telephone number is (571) 270-7752 and fax number is (571) 270-8752. The examiner can normally be reached on M-F 8:30am-5:00pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Fred Ehichioya can be reached on (571) 272-4034.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/RINNA YI/
Primary Examiner, Art Unit 2179
Prosecution Insights