CTNF 18/991,005 CTNF 66123 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claims 1-20 are rejecte d under 3 5 U.S.C. 103 as being unpatentable over ROSENBERG et al ( 20170031502). As per claim 1, Ros enberg teaches the claimed “interaction method,” wherein, the method comprises: “displaying a virtual object in a virtual reality environment” ( Rosenberg, [0054] - Generally, in Block S150, the computer system manipulates a virtual surface (or a virtual object) within the virtual environment according to the force magnitude and direction of a virtual force vector generated in Block S14 ); “mapping a real interactor to the virtual reality environment to form a virtual interactor, based on a real position of the real interactor” ( Rosenberg, [0059] - In this example application, the computer system: generates digital (two- or three-dimensional) frames of a virtual environment including a potter's wheel, a virtual mass of clay, and virtual hands ; and translates a force profile across a contact area of an input object on the touch sensor surface into a virtual finger (or fingers, palm, hand) that works the virtual clay mass spinning on the virtual potter's wheel ); “determining an offset between a virtual position and the real position based on an interaction operation between the real interactor and the virtual object in the virtual reality environment” ( Rosenberg, [0059] - For example, as a user depresses a thumb on the first touch sensor, the computer system can update frames of the virtual environment to show a virtual thumb (or first virtual forming tool) pushing the virtual clay mass inward toward the center of the virtual potter's wheel according to the position and force magnitude of the thumb input and the position and orientation of the first touch sensor in the real space ) ( Noted : Rosenberg’s position of the virtual thumb, moving inward toward the center of the virtual potter's wheel, is offset to the real position of the thumb which on the touch sensor surface; in other words, the offset is distance between the virtual inward-moving position of the virtual thumb on display and the real position of the thumb which applies a force on the touch sensor surface, but does not move); and “adjusting the virtual position of the virtual interactor based on the offset to show a position offset state between the virtual position and the real position” ( Rosenberg, [0059] - Similarly, as the user depresses an index finger on the second touch sensor—on the opposite side of the input device—the computer system can update frames of the virtual environment to show a virtual index finger (or second virtual forming tool) contacting the virtual mass of clay, forming a virtual interior wall in the mass of virtual clay spinning on the virtual potter's wheel, and pulling the virtual clay wall outward from the center of the virtual potter's wheel. In this example, as the user depresses a thumb onto the first touch sensor and an index finger on the second touch sensor (i.e., as the user pinches the input device between the first and second touch sensors), the computer system can generate subsequent digital frames of the virtual environment that depict virtual fingers compressing the virtual wall on the potter's wheel according to a measured compressive force applied by the user across the first and second touch sensor surfaces ). Thus, it would have been obvious to configure Rosenberg’s method as claimed by adjusting the virtual positions of the virtual thumb and index finger, inwardly and upwardly, based on the offsets according to the position and force magnitude of the thumb input and the position and orientation of the first touch sensor in the real space. The motivation is to perform operation and interaction more naturally and intuitively. Claim 2 adds into claim 1 “wherein the determining an offset between a virtual position and a real position based on an interaction operation between the real interactor and the virtual object in the virtual reality environment comprises: determining, based on the interaction operation between the real interactor and the virtual object in the virtual reality environment, a resistance of the virtual object relative to the virtual interactor; and determining the offset between the virtual position and the real position based on the resistance” ( Rosenberg, [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object ). Claim 3 adds into claim 2 “wherein the determining, based on the interaction operation between the real interactor and the virtual object in the virtual reality environment, a resistance of the virtual object relative to the virtual interactor comprises: obtaining a motion direction, a depth, and an acceleration that are generated when the real interactor performs an interaction operation with the virtual object in the virtual reality environment, the depth being a depth value of the real interactor relative to a virtual surface of the virtual object, and the acceleration being an acceleration of the real interactor moving relative to the virtual object” ( Rosenberg, [0018 ] - A computer system executing the first method S100 can fuse the force magnitude of an input on a touch sensor surface, an orientation of the touch sensor surface in real space, and the location of the input on the touch sensor surface into: a force vector (i.e., a force magnitude and a direction) ; and an origin of the force vector within the virtual environment. The computer system can then project the force vector into a virtual environment to define a magnitude and direction of a change in the position, size, or geometry, etc. of a virtual object and/or a virtual surface within the virtual environment , thereby linking user inputs on the touch sensor surface in physical (i.e., real) space to control or manipulate virtual objects or virtual surfaces within the virtual environment ; Rosenberg, [0055] - the computer system moves and/or deforms a virtual surface (or a virtual object) within the virtual model according to a virtual force vector and a physics model defining mechanical creep, elastic deformation, plastic deformation, and/or inertial dynamics, etc. of the virtual surface and/or the virtual object . In particular, the computer system can implement the physical model to: calculate deformation of a virtual surface mapped to the touch sensor surface; calculate changes in the position and orientation of a virtual object; and calculate inertial effects on the virtual object resulting from a virtual force—of magnitude, direction, and position (e.g., origin) defined in the virtual force vector—applied to the virtual surface over the duration of one scan period; [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object) ( Noted: Rosenberg’s force vector defining a magnitude and direction of a change in the position , size, or geometry, etc. of a virtual object and/or a virtual surface within the virtual environment hypothetically illustrates a motion acceleration, and a moving depth of the thumb); and “determining the resistance of the virtual object relative to the virtual interactor based on the motion direction, the depth, and the acceleration” ( Rosenberg, [0055] - the computer system moves and/or deforms a virtual surface (or a virtual object) within the virtual model according to a virtual force vector and a physics model defining mechanical creep, elastic deformation, plastic deformation, and/or inertial dynamics, etc. of the virtual surface and/or the virtual object . In particular, the computer system can implement the physical model to: calculate deformation of a virtual surface mapped to the touch sensor surface; calculate changes in the position and orientation of a virtual object; and calculate inertial effects on the virtual object resulting from a virtual force—of magnitude, direction, and position (e.g., origin) defined in the virtual force vector—applied to the virtual surface over the duration of one scan period; [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object ). Claim 4 adds into claim 3 “determining the resistance value of the virtual object relative to the virtual interactor based on the depth and the acceleration, wherein the depth is in a direct proportion to the resistance value, and the acceleration is in a direct proportion to the resistance value” ( Rosenberg, [0055] - the computer system moves and/or deforms a virtual surface (or a virtual object) within the virtual model according to a virtual force vector and a physics model defining mechanical creep, elastic deformation, plastic deformation, and/or inertial dynamics, etc. of the virtual surface and/or the virtual object . In particular, the computer system can implement the physical model to: calculate deformation of a virtual surface mapped to the touch sensor surface; calculate changes in the position and orientation of a virtual object; and calculate inertial effects on the virtual object resulting from a virtual force—of magnitude, direction, and position (e.g., origin) defined in the virtual force vector—applied to the virtual surface over the duration of one scan period; [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object ); and determining the resistance direction of the virtual object relative to the virtual interactor based on the motion direction of the real interactor, the resistance direction being opposite to the motion direction” ( Rosenberg, [0044] - Generally, in Blocks S140 and S142, the computer system combines a magnitude of a force applied to the touch sensor surface with an orientation of the touch sensor surface within the real space (within a sampling or scan period) and the location of the applied force on the touch sensor surface to generate a virtual force vector representing a magnitude, direction, and origin of a virtual force to be applied to a virtual surface (or to the virtual object generally) within the virtual environment to manipulate the virtual object in Block S150 ) ( Noted: the resistance force is, theoretically, opposite to the motion direction). Claim 5 adds into claim 3 “obtaining a physical characteristic parameter of the virtual object, the physical characteristic parameter comprising at least one of mass, material hardness, and volume; and determining the resistance value of the virtual object relative to the virtual interactor based on the physical characteristic parameter of the virtual object, the depth, and the acceleration, a parameter value of the physical characteristic parameter being in a direct proportion to the resistance value” ( Rosenberg, [0055] - the computer system moves and/or deforms a virtual surface (or a virtual object) within the virtual model according to a virtual force vector and a physics model defining mechanical creep, elastic deformation, plastic deformation, and/or inertial dynamics, etc. of the virtual surface and/or the virtual object . In particular, the computer system can implement the physical model to: calculate deformation of a virtual surface mapped to the touch sensor surface; calculate changes in the position and orientation of a virtual object; and calculate inertial effects on the virtual object resulting from a virtual force—of magnitude, direction, and position (e.g., origin) defined in the virtual force vector—applied to the virtual surface over the duration of one scan period; [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object ). Claim 6 adds into claim 4 “determining the displacement distance of the offset based on the resistance value, the resistance value being in a direct proportion to the displacement distance” ( Rosenberg, [0055] - the computer system moves and/or deforms a virtual surface (or a virtual object) within the virtual model according to a virtual force vector and a physics model defining mechanical creep, elastic deformation, plastic deformation, and/or inertial dynamics, etc. of the virtual surface and/or the virtual object . In particular, the computer system can implement the physical model to: calculate deformation of a virtual surface mapped to the touch sensor surface; calculate changes in the position and orientation of a virtual object; and calculate inertial effects on the virtual object resulting from a virtual force—of magnitude, direction, and position (e.g., origin) defined in the virtual force vector—applied to the virtual surface over the duration of one scan period; [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object ) ( Noted: When a material is deformed (stretched, compressed, or sheared) within its elastic limit, the stress (force per unit area) is proportional to the strain (fractional change in length or shape) ); and determining the displacement direction of the offset based on the motion direction of the real interactor, the motion direction being the same as the displacement direction” ( Rosenberg, [0044] - Generally, in Blocks S140 and S142, the computer system combines a magnitude of a force applied to the touch sensor surface with an orientation of the touch sensor surface within the real space (within a sampling or scan period) and the location of the applied force on the touch sensor surface to generate a virtual force vector representing a magnitude, direction, and origin of a virtual force to be applied to a virtual surface (or to the virtual object generally) within the virtual environment to manipulate the virtual object in Block S150 ) ( Noted: the motion direction is, theoretically, being the same as the displacement direction of the virtual surface). Claim 7 adds into claim 6 “wherein the adjusting the virtual position of the virtual interactor based on the offset comprises: when the resistance value is less than a first resistance threshold, changing the virtual position of the virtual interactor based on the offset and a position change of the real interactor; or when the resistance value is greater than or equal to the first resistance threshold, controlling the virtual position of the virtual interactor to remain unchanged in the virtual reality environment” ( Rosenberg, [0044] - Generally, in Blocks S140 and S142, the computer system combines a magnitude of a force applied to the touch sensor surface with an orientation of the touch sensor surface within the real space (within a sampling or scan period) and the location of the applied force on the touch sensor surface to generate a virtual force vector representing a magnitude, direction, and origin of a virtual force to be applied to a virtual surface (or to the virtual object generally) within the virtual environment to manipulate the virtual object in Block S150 ) ( Noted: When an applied force is less than the surface stress, the material’s surface does not deform because the external load is insufficient to overcome the internal restoring forces that maintain the surface’s original shape). Claim 8 adds into claim 4 “determining the posture offset amplitude of the offset based on the resistance value; and determining the posture offset direction of the offset based on the resistance direction, the resistance direction being the same as the posture offset direction” ( Rosenberg, [0055] - the computer system moves and/or deforms a virtual surface (or a virtual object) within the virtual model according to a virtual force vector and a physics model defining mechanical creep, elastic deformation, plastic deformation, and/or inertial dynamics, etc. of the virtual surface and/or the virtual object . In particular, the computer system can implement the physical model to: calculate deformation of a virtual surface mapped to the touch sensor surface; calculate changes in the position and orientation of a virtual object; and calculate inertial effects on the virtual object resulting from a virtual force—of magnitude, direction, and position (e.g., origin) defined in the virtual force vector—applied to the virtual surface over the duration of one scan period; [0057] - The computer system can also “stretch” a peripheral region of the surface around the deformed region as the user continues to apply a force to the touch sensor surface over a series of consecutive scan periods, such as based on a modulus of elasticity defined in a physical model assigned to the virtual object ) ( Noted: The posture offset is defined as the visual deformation of the virtual surface, therefore, the posture offset amplitude and direction are corresponding to the applied force, or the resistance force). Claim 9 adds into claim 8 “wherein the determining the posture offset amplitude of the offset based on the resistance value comprises: when the resistance value is less than a second resistance threshold, determining the posture offset amplitude of the offset based on the resistance value, the resistance value being in a direct proportion to the posture offset amplitude; or when the resistance value is greater than or equal to the second resistance threshold, determining that the posture offset amplitude of the offset remains unchanged based on the second resistance threshold” ( Rosenberg, [0044] - Generally, in Blocks S140 and S142, the computer system combines a magnitude of a force applied to the touch sensor surface with an orientation of the touch sensor surface within the real space (within a sampling or scan period) and the location of the applied force on the touch sensor surface to generate a virtual force vector representing a magnitude, direction, and origin of a virtual force to be applied to a virtual surface (or to the virtual object generally) within the virtual environment to manipulate the virtual object in Block S150 ) ( Noted: When an applied force is less than the surface stress, the material’s surface does not deform because the external load is insufficient to overcome the internal restoring forces that maintain the surface’s original shape; therefore, since the posture offset is defined as the visual deformation of the virtual surface, the posture offset amplitude and direction are corresponding to the applied force, or the resistance force). Claim 10 adds into claim 2 “wherein the method further comprises: determining a color attribute parameter of the virtual interactor based on the resistance, the color attribute parameter comprising at least one of hue, saturation, and lightness; and when the virtual position of the virtual interactor is adjusted based on the offset, adjusting a display color of the virtual interactor based on the color attribute parameter” ( Rosenberg, [0059] - For example, as a user depresses a thumb on the first touch sensor, the computer system can update frames of the virtual environment to show a virtual thumb (or first virtual forming tool) pushing the virtual clay mass inward toward the center of the virtual potter's wheel according to the position and force magnitude of the thumb input and the position and orientation of the first touch sensor in the real space ) ( Noted: Rosenberg’s visual change of the virtual thumb’s position during deforming the virtual surface suggests a conventional color code for visual representation of a display color of the virtual thumb based on the distance of pushing/pulling the virtual surface). Claims 11-15 and 16-20 claim a non-transitory computer-readable storage medium and a terminal device based on the method of claims 1-10; therefore, they are rejected under a similar rationale. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHU K NGUYEN whose telephone number is (571)272-7645. The examiner can normally be reached M-F 8-5pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7 642 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /PHU K NGUYEN/Primary Examiner, Art Unit 2616 Application/Control Number: 18/991,005 Page 2 Art Unit: 2616 Application/Control Number: 18/991,005 Page 3 Art Unit: 2616 Application/Control Number: 18/991,005 Page 4 Art Unit: 2616
Prosecution Insights