DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-13, 49-50, and 53-57 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al. (US Pub: 2019/0332182 A1) in view of Rho et al. (US Pub: 2023/0154060 A1).
As to claim 1, Shen discloses a data processing method (i.e. as seen in figure 1-3 embodiment Shen discloses a data processing method for gesture object input in a virtual reality setting) (see Fig. 1-3, [0066-0086]), comprising:
determining an action position (i.e. the action position of first VR handle 161 as seen in figure 1A and 1B) (see Fig. 1, [0066-0068]) on the basis of a first action signal (i.e. the system of Shen determines a first gesture which is an action position based digital object that is displace in the virtual reality environment at the cursor input area) (see Fig. 2-5, [0083-0086]);
determining a second action posture (i.e. the action posture of second VR handle 162 as seen in figure 1A and 1B) (see Fig. 1, [0066-0068]) on the basis of a second action signal from a second device different from the first device (i.e. as seen in figure 13-14, Shen teaches the remote control unit that is separate from the HMD display device which is able to input the user’s hand movement in the three degree of freedom control to indicate user’s hand gestures) (see Fig. 12-13, [0131-0136]), wherein the second action posture indicates a body action posture of a user (i.e. the user’s hand gestures based on user’s movement) (see Fig. 12-13, [0131-0134]), and the second action signal is a three-degree of freedom action signal (i.e. the second gesture of Shen uses the action posture of the user input signal from the input device which is a ray that is explicitly display on the display) (see Fig. 2-5, 12-13, [0083-0095], [0131-0134]); and
executing an operation instruction based on the action position and the second action posture and a second action signal (i.e. as seen in figures 1-5, 12-13 the system of Shen displays a virtual reality object being interacted upon from the user gesture to create direct interaction in the display objects where the HMD device and the remote devices functions together where the operation are based on both the HMD action position and the remote action position) (see Fig. 1-5, 12-13 [0066-00103] and [0131-0136])).
However, Shen does not teach a first action signal from a first device, wherein the action position indicates a corresponding position of the first device in a virtual reality scene, and the first action signal is a six-degree-of-freedom action signal (i.e. Shen teaches an HMD with virtual reality function but do not describe a six-degree-of-freedom-action signal).
Rho teaches a first action signal from a first device, wherein the action position indicates a corresponding position of the first device in a virtual reality scene, and the first action signal is a six-degree-of-freedom action signal (i.e. as seen in figure 1-2, the camera system of the HMD device 200 is described as capable of performing special recognition of 6 degree of freedom for the purposes of input control for the user) (see Fig. 1-2, [0058-0059]).
Since both Shen and Rho teaches an HMD style virtual reality display system where user interactions is used to control objects, they are analogous as having the same field of endeavor.
Therefore, it would have been obvious for one of ordinary skill in the art at the accepted filing date of the current invention to have used the 6 degree of freedom action signal of Rho’s HMD device in the overall HMD and control system of Shen to extend the control capacity of Shen to enable a further ergonomical control means for the user to interact with the virtual environment with an actual deployable strategy to implement device control on a physical level as Shen is silent with respect to the ways by which the device can do spatial recognition to implement the 3D environment (see Rho, Fig. 1-3, [0057-0059]).
As to claim 49, Shen teaches an electronic device (i.e. as seen in figure 1-3 and 12-13 embodiments Shen discloses a head mounted display device for gesture object input in a virtual reality setting) (see Fig. 1-3, 12-13, [0066-0086] and [0131-0135]), comprising:
a processor (i.e. the processor shown in figure 29 and 30) (see Fig. 29-30, [0291-0293]);
and a memory (i.e. element 144 of figure 29 embodiment) (see Fig. 29) configured to store instructions executable by the processor, the processor being configured to read the executable instruction from the memory and execute the instructions to implement a data processing (i.e. the system of figure 1-5 uses a computer to implement the virtual reality display system) (see Fig. 1-5, 29-30, [0291-0294]) method comprising:
determining an action position (i.e. the action position of first VR handle 161 as seen in figure 1A and 1B) (see Fig. 1, [0066-0068]) on the basis of a first action signal (i.e. the system of Shen determines a first gesture which is an action position based digital object that is displaced in the virtual reality environment at the cursor input area) (see Fig. 2-5, [0066-0086]);
determining a second action posture (i.e. the action posture of second VR handle 162 as seen in figure 1A and 1B) (see Fig. 1, [0066-0068]) on the basis of a second action signal (i.e. the second gesture of Shen uses the action posture of the user input signal from the input device which is a ray that is explicitly display on the display) (see Fig. 1-5, [0083-0095]) on the basis of a second action signal from a second device different from the first device (i.e. as seen in figure 13-14, Shen teaches the remote control unit that is separate from the HMD display device which is able to input the user’s hand movement in the three degree of freedom control to indicate user’s hand gestures) (see Fig. 12-13, [0131-0136]), wherein the second action posture indicates a body action posture of a user (i.e. the user’s hand gestures based on user’s movement) (see Fig. 12-13, [0131-0134]), and the second action signal is a three-degree of freedom action signal (i.e. the second gesture of Shen uses the action posture of the user input signal from the input device which is a ray that is explicitly display on the display) (see Fig. 2-5, 12-13, [0083-0095], [0131-0134]); and
executing an operation instruction based on the action position and the second action posture (i.e. as seen in figures 1-5, 12-13 the system of Shen displays a virtual reality object being interacted upon from the user gesture to create direct interaction in the display objects where the HMD device and the remote devices functions together where the operation are based on both the HMD action position and the remote action position) (see Fig. 1-5, 12-13 [0066-00103] and [0131-0136])).
However, Shen does not teach a first action signal from a first device, wherein the action position indicates a corresponding position of the first device in a virtual reality scene, and the first action signal is a six-degree-of-freedom action signal (i.e. Shen teaches an HMD with virtual reality function but do not describe a six-degree-of-freedom-action signal).
Rho teaches a first action signal from a first device, wherein the action position indicates a corresponding position of the first device in a virtual reality scene, and the first action signal is a six-degree-of-freedom action signal (i.e. as seen in figure 1-2, the camera system of the HMD device 200 is described as capable of performing special recognition of 6 degree of freedom for the purposes of input control for the user) (see Fig. 1-2, [0058-0059]).
Since both Shen and Rho teaches an HMD style virtual reality display system where user interactions is used to control objects, they are analogous as having the same field of endeavor.
Therefore, it would have been obvious for one of ordinary skill in the art at the accepted filing date of the current invention to have used the 6 degree of freedom action signal of Rho’s HMD device in the overall HMD and control system of Shen to extend the control capacity of Shen to enable a further ergonomical control means for the user to interact with the virtual environment with an actual deployable strategy to implement device control on a physical level as Shen is silent with respect to the ways by which the device can do spatial recognition to implement the 3D environment (see Rho, Fig. 1-3, [0057-0059]).
As to claim 50, Shen teaches a non-transitory computer-readable storage medium having thereon stored a computer program for performing a data processing method (i.e. as seen in figure 1-3 and 12-13 embodiments Shen discloses a head mounted display device for gesture object input in a virtual reality setting) (see Fig. 1-3, 12-13, [0066-0086] and [0131-0135]) comprising:
determining an action position (i.e. the action position of first VR handle 161 as seen in figure 1A and 1B) (see Fig. 1, [0066-0068]) on the basis of a first action signal (i.e. the system of Shen determines a first gesture which is an action position based digital object that is displaced in the virtual reality environment at the cursor input area) (see Fig. 2-5, [0083-0086]);
determining a second action posture (i.e. the action posture of second VR handle 162 as seen in figure 1A and 1B) (see Fig. 1, [0066-0068]) on the basis of a second action signal (i.e. the second gesture of Shen uses the action posture of the user input signal from the input device which is a ray that is explicitly display on the display) (see Fig. 1-5, [0066-0095]) on the basis of a second action signal from a second device different from the first device (i.e. as seen in figure 13-14, Shen teaches the remote control unit that is separate from the HMD display device which is able to input the user’s hand movement in the three degree of freedom control to indicate user’s hand gestures) (see Fig. 12-13, [0131-0136]), wherein the second action posture indicates a body action posture of a user (i.e. the user’s hand gestures based on user’s movement) (see Fig. 12-13, [0131-0134]), and the second action signal is a three-degree of freedom action signal (i.e. the second gesture of Shen uses the action posture of the user input signal from the input device which is a ray that is explicitly display on the display) (see Fig. 2-5, 12-13, [0083-0095], [0131-0134]); and
executing an operation instruction based on the action position and the second action posture (i.e. as seen in figures 1-5, 12-13 the system of Shen displays a virtual reality object being interacted upon from the user gesture to create direct interaction in the display objects where the HMD device and the remote devices functions together where the operation are based on both the HMD action position and the remote action position) (see Fig. 1-5, 12-13 [0066-00103] and [0131-0136])).
on the basis of a second action signal from a second device different from the first device (i.e. as seen in figure 13-14, Shen teaches the remote control unit that is separate from the HMD display device which is able to input the user’s hand movement in the three degree of freedom control to indicate user’s hand gestures) (see Fig. 12-13, [0131-0136]), wherein the second action posture indicates a body action posture of a user (i.e. the user’s hand gestures based on user’s movement) (see Fig. 12-13, [0131-0134]), and the second action signal is a three-degree of freedom action signal (i.e. the second gesture of Shen uses the action posture of the user input signal from the input device which is a ray that is explicitly display on the display) (see Fig. 2-5, 12-13, [0083-0095], [0131-0134]).
However, Shen does not teach a first action signal from a first device, wherein the action position indicates a corresponding position of the first device in a virtual reality scene, and the first action signal is a six-degree-of-freedom action signal (i.e. Shen teaches an HMD with virtual reality function but do not describe a six-degree-of-freedom-action signal).
Rho teaches a first action signal from a first device, wherein the action position indicates a corresponding position of the first device in a virtual reality scene, and the first action signal is a six-degree-of-freedom action signal (i.e. as seen in figure 1-2, the camera system of the HMD device 200 is described as capable of performing special recognition of 6 degree of freedom for the purposes of input control for the user) (see Fig. 1-2, [0058-0059]).
Since both Shen and Rho teaches an HMD style virtual reality display system where user interactions is used to control objects, they are analogous as having the same field of endeavor.
Therefore, it would have been obvious for one of ordinary skill in the art at the accepted filing date of the current invention to have used the 6 degree of freedom action signal of Rho’s HMD device in the overall HMD and control system of Shen to extend the control capacity of Shen to enable a further ergonomical control means for the user to interact with the virtual environment with an actual deployable strategy to implement device control on a physical level as Shen is silent with respect to the ways by which the device can do spatial recognition to implement the 3D environment (see Rho, Fig. 1-3, [0057-0059]).
As to claim 2, Shen and Rho teaches the data processing method according to claim 1, wherein the method satisfies at least one of the following: the first action signal is a six-degree-of-freedom action signal (i.e. as seen in figure 1-2 of Rho, the camera system of the HMD device 200 is described as capable of performing special recognition of 6 degree of freedom for the purposes of input control for the user that is separate from the users hand gesture control as seen in Shen) (see Shen Fig. 1-3, and Rho Fig. 1-2, [0058-0059]).
As to claim 3, Shen and Rho the data processing method according to claim 1, wherein the method further comprises: determining a first action posture on the basis of the first action signal; and the executing the operation instruction based on the action position and the second action posture, comprises: executing the operation instruction based on the action position, the first action posture, and the second action posture (i.e. as seen in figure 1-5 the Shen embodiment discloses a multiple input sensing virtual reality system which tracks first and second gesture to display a direct manipulation on a virtual objects which is based on first and second action signal from the input of the virtual hand and the actual position of the display of the HMD that is worn by the user) (see Fig. 1-5, [066-0098]).
As to claim 4, Shen and Rho the data processing method according to claim 1, wherein the method further comprises: determining the first action signal based on action data of a first device; and determining the second action signal based on action data of a second device (i.e. the first device the is the handle unit 122 which detect the user hand movement, while the second device is the HMD device 123 as seen in figure 12) (see Fig. 12, [0131-0132]).
As to claim 5, Shen and Rho the data processing method according to claim 1, wherein the executing an operation instruction based on the action position and the second action posture, comprises: executing a first operation instruction based on the action position; and executing a second operation instruction based on the second action posture (i.e. as seen in figure 1B the Shen disclosure shows a virtual reality device where the position of the virtual handle unit 161 is tracked and the second action posture is from the second VR handle 162 as seen in figure 1B where the first and second hand is held by the user’s left and right hand) (see Fig. 1B, [0066-0067]).
As to claim 6, Shen and Rho the data processing method according to claim 5, wherein the first operation instruction corresponds to first display content of a display device, the first display content being determined based on a position and posture of the display device; and the second operation instruction corresponds to second display content of the display device, the second display content being determined based on the posture of the display device, or the second display content being display content at a preset position (i.e. the device of Shen shows the virtual reality environment where the HMD display device of figure 12 display many virtual objects which is interacted upon by the control handle 122 where the posture of the virtual object is displayed on the HMD device) (see Fig. 12, [0096-0098]).
As to claim 7, Shen and Rho teaches the data processing method according to claim 5, wherein the executing the first operation instruction based on the action position, comprises: in response to the action position being matched with a first operation position of a first target object, playing an effect corresponding to the first target object (i.e. as seen in figure 1-5 the user is able to interact with the virtual object on the HMD device and active the object with the control of the VR handle 161 as seen in figure 1B) (see Fig. 1-5, [0066-0069]); the method further comprises: determining a second action time on the basis of the second action signal (i.e. as seen in figure 20 and 21 the interactive virtual menu system of Shen shows how the user is able to interact with the VR handle and control the interface at specific time for activating the function of figure 20 embodiment) (see Fig. 1-5, 20-21, [0231-0234]); and the executing the second operation instruction based on the second action posture, comprises: in response to the second action time being matched with an operation effective time of a second target object, and the second action posture being matched with a second operation position of the second target object, playing an effect corresponding to the second target object (i.e. as seen in figure 20 and 21 the interactive virtual menu system of Shen shows how the user is able to activate a deeper level of the spatial menu position to reach the second level menu with the VR handle and control the interface at specific time for activating the function of figure 20-21 embodiments) (see Fig. 1-5, 20-21, [0231-0234]).
As to claim 8, Shen and Rho teaches the data processing method according to claim 5, wherein the executing the first operation instruction based on the action position (i.e. as seen in figure 21-22 the first level menu that is virtual input can reach as seen in figure 21) (see Fig. 21, [0233]), comprises: in response to the action position being matched with an operation trajectory within an operation effective time, playing a first operation effect (i.e. the HMD display the menu icon on the display interface) (see Fig. 20-21, [0231-0233]); and the executing the second operation instruction based on the second action posture (i.e. the user is able to use the second display handle 162 of figure 1B to activate the second level menu as shown in figure 21) (see Fig. 1B, 21, [0231-0232]), comprises: in response to the second action posture meeting a preset condition within the operation effective time, playing a second operation effect (i.e. the hand position detection of the figure 21 embodiment detects the second VR handle 162 active activate the second level menu) (see Fig. 1B, 20-21, [0066-0068], [0231-0233]).
As to claim 9, Shen and Rho teaches the data processing method according to claim 8, wherein the method further comprises: displaying an operation prompt identification within the operation effective time (i.e. as seen in figure 20-21 the menu as display on the HMD has a time delay effect has the computer generate different level of user interface that changes with each user interaction within the operation effective time as seen in the flow chart of figure 21) (see Fig. 20-21, [0231-0235]).
As to claim 10, Shen and Rho teaches the data processing method according to claim 8, wherein the method further comprises: displaying a position identification associated with the action position (i.e. the display shows the cursor on the HMD which activate the display icon of the menu as seen in figure 20) (see Fig. 20, [0231-0232]).
As to claim 11, Shen and Rho teaches the data processing method according to claim 1, wherein the executing the operation instruction based on the second action posture and the action position, comprises: executing the operation instruction corresponding to the action position and the second action posture (i.e. the embodiment of figure 20-21 is based on the hardware system of figure 1B which shows that both the first and second VR hand 161 and 162 activate the control of the HMD device) (see Fig. 1B, 20-21, [0066-0068], [0231-0235]).
As to claim 12, Shen and Rho teaches the data processing method according to any of claim11, wherein the executing the operation instruction corresponding to the action position and the second action posture, comprises: when the second action posture meets a preset condition, executing an operation instruction corresponding to the action position; or the method further comprises: determining a first action posture on the basis of the first action signal; and the executing the operation instruction corresponding to the action position and the second action posture, comprises: when the second action posture meets a preset condition, executing an operation instruction corresponding to the first action posture and the action position (i.e. as seen in figure 20 and 21 the interactive virtual menu system of Shen shows how the user is able to activate a deeper level of the spatial menu position to reach the second level menu with the VR handle and control the interface at specific time for activating the function of figure 20-21 embodiments) (see Fig. 1-5, 20-21, [0231-0234]).
As to claim 13, Shen and Rho teaches the data processing method according to claim 11, wherein the executing the operation instruction corresponding to the action position and the second action posture, comprises: executing the operation instruction corresponding to the action position, and displaying a real-time execution result; and when the second action posture meets a preset condition, obtaining an execution result at a current time (i.e. the execution of the real-time feedback input output system of Shen shows the VR handle directly inface with the HMD device through a computer as seen in figure 1A 1B embodiment which means that both the first 161 VR handle and the second 162 VR handle both generate control that access the display menu system of figure 20 and 21) (see Fig. 1A, 1B, 20 and 21, [0066-0068], 0231-0235]).
As to claim 53, Shen and Rho teaches the electronic device according to claim 49, wherein the method satisfies at least one of the following: the first action signal is a six-degree-of-freedom action signal (i.e. the system of Shen in figure 1B and 12-13 shows a virtual reality display system with direction virtual object manipulation control of the virtual handle having a 3D virtual reality space with the movement and rotational direction of the real hand is tracked by the input device 161) (see Fig. 1B, 12-13, [0066-0068], [0095-0098]); or the second action signal is a three-degree-of-freedom action signal (i.e. the second action signal can be a positional detection for point of reference as seen in figure 1B VR handle 162 where the actual gesture is detected for ray display on the virtual environment) (see Fig. 2-5, and 12-13, [0083-0098]).
As to claim 54, Shen and Rho teaches the electronic device according to claim 49, wherein the method further comprises: determining a first action posture on the basis of the first action signal; and the executing the operation instruction based on the action position and the second action posture, comprises: executing the operation instruction based on the action position, the first action posture, and the second action posture (i.e. as seen in figure 20 and 21 the interactive virtual menu system of Shen shows how the user is able to activate a deeper level of the spatial menu position to reach the second level menu with the VR handle and control the interface at specific time for activating the function of figure 20-21 embodiments) (see Fig. 1-5, 20-21, [0231-0234]).
As to claim 55, Shen and Rho teaches the electronic device according to claim 49, wherein the method further comprises: determining the first action signal based on action data of a first device; and determining the second action signal based on action data of a second device(i.e. as seen in figure 20 and 21 the interactive virtual menu system of Shen shows how the user is able to activate a deeper level of the spatial menu position to reach the second level menu with the VR handle and control the interface at specific time for activating the function of figure 20-21 embodiments) (see Fig. 1-5, 20-21, [0231-0234]).
As to claim 56, Shen and Rho teaches the electronic device according to claim 49, wherein the executing an operation instruction based on the action position and the second action posture, comprises: executing a first operation instruction based on the action position; and executing a second operation instruction based on the second action posture (i.e. as seen in figure 20 and 21 the interactive virtual menu system of Shen shows how the user is able to activate a deeper level of the spatial menu position to reach the second level menu with the VR handle and control the interface at specific time for activating the function of figure 20-21 embodiments) (see Fig. 1-5, 20-21, [0231-0234]).
As to claim 57, Shen and Rho teaches the electronic device according to claim 56, wherein the first operation instruction corresponds to first display content of a display device, the first display content being determined based on a position and posture of the display device; and the second operation instruction corresponds to second display content of the display device, the second display content being determined based on the posture of the display device, or the second display content being display content at a preset position (i.e. the execution of the real-time feedback input output system of Shen shows the VR handle directly inface with the HMD device through a computer as seen in figure 1A 1B embodiment which means that both the first 161 VR handle and the second 162 VR handle both generate control that access the display menu system of figure 20 and 21) (see Fig. 1A, 1B, 20 and 21, [0066-0068], 0231-0235]).
Response to Arguments
Applicant’s arguments with respect to claims 1, 3-13, 49-50, and 53-57 have been considered but are moot because the new reference Rho applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art Jeong (US Patent 11,914,835 B2) is cited to teach another type of HMD input device with six degree of freedom control in figure 1-5 embodiments.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CALVIN C. MA whose telephone number is (571)270-1713. The examiner can normally be reached 8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benjamin C. Lee can be reached on 571-272-2963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CALVIN C MA/Primary Examiner, Art Unit 2693 June 7, 2026
Prosecution Insights