DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10 and 11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10 recites the limitation "the predetermined action sequence" in line 2. There is insufficient antecedent basis for this limitation in the claim. Appropriate correction is required.
Claim 11 recites the following:
“wherein any of the first predetermined action sequence and the second predetermined action sequence comprises an action sequence performed by any of: an arm, two hands, and a single hand, and an ending action in the first predetermined action sequence is the same as a starting action in the second predetermined action sequence”.
The bold portion is unclear based on the context of the recitation. The bold portion can be interpreted as one of the four causes of an action sequence, or as a self-contained feature. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-8, 11-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 2022/0091722 to Faulkner et al. (hereinafter Faulkner).
Regarding claims 1, 15, and 20, Faulkner teaches an electronic device comprising:
at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit (Faulkner, paragraph [0012], line 8-13, “one or more processors, and memory storing one or more programs; the one or more programs are configured to be executed by the one or more processors and the one or more programs include instructions for performing or causing performance of the operations of any of the methods described herein”), causing the electronic device to perform acts for controlling a virtual object in an application (Faulkner, paragraph [0037], line 18-24, “Subsequently, the computer system decouples the position of the virtual object from the location of the user's hand, so that the user can use the same hand (and optionally, using a different hand, or a combination of both hands, etc.) to provide gestures to interact with the virtual object and the controls contained therein”), the acts comprising:
in response to a determination that a first action sequence of a user of the application matches a first predetermined action sequence, presenting, in a display area of the application, a set of virtual objects with a first format (Faulkner, figure 7 E-F; paragraph [0037], line 14-18, “Accordingly, the user can influence the initial position of the virtual object in the three-dimensional environment by choosing the location of his/her hand when providing the required movement of the hand to trigger display of the virtual object”);
determining, based on a user interaction of the user, a target virtual object in the set of virtual objects (Faulkner, figure 7 G-H, the virtual object 7342 of a list of virtual objects is highlighted when a user’s hand virtually touches the virtual object 7342); and
in response to a determination that a second action sequence of the user matches a second predetermined action sequence, performing an operation corresponding to the target virtual object (Faulkner, paragraph [0037], line 14-18, when the user provides the required movement of the hand (an action sequence), the user triggers display of the virtual object (an operation is performed)).
Further regarding claim 1, Faulkner teaches a method of controlling a virtual object in an application (Faulkner, paragraph [0037], line 18-24).
Further regarding claim 20, Faulkner teaches a non-transitory computer readable storage medium having a computer program stored thereon, the computer program, when executed by a processor (Faulkner, paragraph [0272], line 9-13, “In some embodiments, the method 9000 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a computer system”), causing the processor to implement acts for controlling a virtual object in an application (Faulkner, paragraph [0037], line 18-24).
Regarding claims 2 and 16, Faulkner teaches presenting the first action sequence in the display area, wherein a presentation position of the set of virtual objects in the display area corresponds to a presentation position of the first action sequence in the display area (Faulkner, paragraph [0037], line 11-18, “The computer system selects the initial position of the virtual object based on the location of the user's hand at the conclusion of the movement that met the preset criteria. Accordingly, the user can influence the initial position of the virtual object in the three-dimensional environment by choosing the location of his/her hand when providing the required movement of the hand to trigger display of the virtual object”).
Regarding claims 3 and 17, Faulkner teaches presenting, in the display area, the target virtual object with a second format different from the first format (Faulkner, figure 7H, the user selects the virtual object 7342 with a second format, and the virtual objects 7336, 7338, and 7340 are a set of virtual objects with a first format).
Regarding claim 4, Faulkner teaches presenting, in the display area, the target virtual object being selected with a third format different from
the first format and the second format (Faulkner, figure 7H, the virtual object 7344 is with a third format).
Regarding claims 5 and 18, Faulkner teaches in response to detecting a third action sequence of the user, performing, based on the third action sequence, the operation corresponding to the target virtual object (Faulkner, figure 7U-V describes a physical rectangular object; figure 7Y-Z, the user holds the physical rectangular object to perform an action sequence and interact with a list of virtual spheres).
Regarding claims 6 and 19, Faulkner teaches in response to a determination that a fourth action sequence of the user matches a third predetermined action sequence, exiting an operation corresponding to the target virtual object (Faulkner, paragraph [0146], line 26-30, “The computer system ceases to move the user interface object 7334 when the user closes his hand or turns his hand such that the palm is no longer facing the user's face, in accordance with some embodiments”).
Regard claim 7, Faulkner teaches wherein the third predetermined action sequence comprises the first predetermined action sequence (Faulkner, paragraph [0225], line 18-21, “In some embodiments, the input that corresponds to the request to display the preview and the input that corresponds to the request to display the computer-generated experience are the same type of input”).
In order to allow a user’s input to perform an outcome, the application needs to have some kinds of predefined input configurations in order to validate the user’s input. In this case, the predetermined action sequences are the application’s predefined input configuration. The first input in the citation stands for the first predetermined action sequence, and the second input in the citation stands for the third predetermined action sequence. Faulkner teaches or suggests that different outcomes are produced by the same input, which concludes that the third predetermined action sequence comprises the first predetermined action sequence.
Regarding claim 8, Faulkner teaches further comprising at least one of:
in response to that no action sequence matching the second predetermined action sequence is detected within a predetermined time period and the user interaction matches none of the set of virtual objects, removing the set of virtual objects from the display area (Faulkner, paragraph [0131], line 25-32, “In some embodiments, after respective representations of the index items are displayed in the index region in accordance with detection of the user's gaze input in the index region, the respective representations of the index items will cease to be displayed if the user's gaze input is no longer detected on the index region or if there are no interactions with the user interface object for a predetermined period of time”).
Regarding claim 11, Faulkner teaches wherein any of the first predetermined action sequence and the second predetermined action sequence comprises an action sequence performed by any of:
an arm, two hands, and a single hand (Faulkner, paragraph [0037], line 18-24, “Subsequently, the computer system decouples the position of the virtual object from the location of the user's hand, so that the user can use the same and (and optionally, using a different hand, or a combination of both hands, etc.) to provide gestures to interact with the virtual object and the controls contained therein”), and
an ending action in the first predetermined action sequence is the same as a starting action in the second predetermined action sequence (Faulkner, paragraph [0279], line 2-7, “in accordance with a determination that the second movement of the first hand corresponds to placing the first hand into a third configuration (e.g., pointing the index finger up, pointing two fingers forward, etc.), the computer system performs a first operation with respect to the first virtual object”).
A person having ordinary skill in the art would know that multiple action sequences can be performed between the first predetermined action sequence and the second predetermined action sequence, even if the ending action in the first predetermined action sequence is the same as a starting action in the second predetermined action sequence.
Regarding claims 12 and 14, Faulkner teaches wherein:
any of the first action sequence and the second action sequence is determined based on data collected by (Faulkner, paragraph [0062], line 1-6, “In some embodiments, the data obtaining unit 242 is configured to obtain data (e.g., presentation data, interaction data, sensor data, location data, etc.) from at least the display generation component 120 of FIG. 1, and optionally one or more of the input devices 125, output devices 155, sensors 190, and/or peripheral devices 195”) at least one of:
an environmental sensor in a wearable device worn by the user, or
an accessory device associated with the wearable device (Faulkner, paragraph [0043], line 11-20, “one or more sensors 190 (e.g., image sensors, light sensors, depth sensors, tactile sensors, orientation sensors, proximity sensors, temperature sensors, location sensors, motion sensors, velocity sensors, etc.), and optionally one or more peripheral devices 195 (e.g., home appliances, wearable devices, etc.). In some embodiments, one or more of the input devices 125, output devices 155, sensors 190, and peripheral devices 195 are integrated with the display generation component 120 (e.g., in a head-mounted device or a handheld device)”);
a gaze direction of the user interaction is determined based on data collected by an eye tracking sensor in the wearable device (Faulkner, paragraph [0089], line 2-7, “In some embodiments, the eye tracking device 130 is controlled by the eye tracking unit 245 (FIG. 2) to track the position and movement of the user's gaze with respect to the scene 105 or with respect to the CGR content displayed via the display generation component 120”); and
the application runs on a computing device in the wearable device (Faulkner, paragraph [0120], line 18-22, “In some embodiments, the input gestures are detected by a sensor system of a head mounted system (e.g., a VR headset that includes a stereoscopic display that provides a left image for the user's left eye and a right image for the user's right eye)”).
Further regarding claim 14, Faulkner teaches wherein the user interaction comprises a user interaction based on at least one of:
a gaze direction of the user (Faulkner, paragraph [0089], line 2-7),
an eye action of the user (Faulkner, paragraph [0089], line 2-7).
Regarding claim 13, Faulkner teaches wherein the action sequence comprises at least one action, and
the action comprises at least one of: an interaction action between the user and the accessory device (Faulkner, paragraph [0183], line 15-21, “when the user is wearing the HMD in the physical environment and wishes to gain access to the home user interface at a later time, the user simply needs to turn toward the selected portion of the physical environment, and the computer system will display the virtual menu at the position of the selected portion of the physical environment”. The user turning toward the selected portion of the physical environment is an action sequence comprising at least one action. The user wearing the HMD and interacting with the virtual menu is an interaction action between the user and the accessory device), or
a body action of at least one body part of the user, the body action being represented by at least one of a posture or a position of the at least one body part of the user (Faulkner, paragraph [0183], line 15-21, the user turning toward the selected portion of the physical environment is also a body action represented by a posture of the user).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Faulkner in view of “Gesture Commands with Kinect Gesture Recognition | by Leyi Sun | Medium” (hereinafter Sun).
Regarding claim 9, Faulkner teaches or suggests the claimed matter as stated above, but does not explicitly teach wherein a predetermined action sequence of the first predetermined action sequence and a predetermined action sequence of the second predetermined action sequence are specified by the user.
Sun teaches wherein a predetermined action sequence of the first predetermined action sequence and a predetermined action sequence of the second predetermined action sequence are specified by the user (Sun, page 1, lines 1-3, “Kinect, a motion sensing device created for Xbox games, enables users to control and interact with their computers using gestures and spoken commands with Windows Kinect software development kit (SDK)”; also see the video “Results”, 00:19 – 00:47).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Faulkner to include the users’ gestures customization process taught by Sun in order to generate more predetermined action sequences. A person having ordinary skill in the art would know that increasing the capacity of the list of predetermined action sequences stored in the system would efficiently produce more approaches to control virtual objects so that users can benefit from customizing the inputs to their personal preferences.
Regarding claim 10, Faulkner in view of Sun teaches or suggests in response to a user interaction for specifying the predetermined action sequence being detected, receiving a video for specifying the predetermined action sequence (Sun, see the video “Record gestures”, Sun recorded a gesture video clip containing a set of action sequences using Microsoft Kinect and the Kinect Studio software); and
determining the predetermined action sequence from the video (Sun, see the video “Analyze the gesture”, 01:43 – 05:22, Sun utilized the Visual Gesture Builder software to obtain a set of action sequences by analyzing the recorded gesture video clip with the predetermined gesture video clip).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Faulkner to include the feature of analyzing gesture video clips to obtain sets of predetermined action sequences taught by Sun in order to generate more predetermined action sequences. A person having ordinary skill in the art would know that increasing the capacity of the list of predetermined action sequences stored in the system would efficiently produce more approaches to control virtual objects so that users can benefit from customizing the inputs to their personal preferences.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHEN-ZHONG LUO whose telephone number is (571)270-5946. The examiner can normally be reached Monday - Friday 7:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Vasat can be reached at (571) 270-7625. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Z.Z.L./Examiner, Art Unit 3715
/WILLIAM H MCCULLOCH JR/Primary Examiner, Art Unit 3715
Prosecution Insights