DETAILED ACTION
This action is responsive to the application filed 7/19/2024.
Claims 1-20 are pending. Claims 3, 4, 6-10, 12-14 and 18 have been amended by preliminary amendment, and Claims 19 and 20 are new.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3, 6, 8, 12, 13, 18 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Osborn, et al., U.S. PGPUB No. 2020/0111260 (“Osborn”).
With regard to Claim 1, Osborn teaches a computer-implemented method for spatially tracking muscle activity, the method comprising:
receiving, from a muscle activation sensor, a muscle activation signal indicating an amount of muscle activation of a muscle associated with a body part ([0171] describes that a plurality of neuromuscular sensors sense and record neuromuscular signals generated by muscle activity at the wrist or forearm);
receiving, from a spatial sensor, a spatial signal indicating a location of the body part in a physical space ([0162] describes that a camera can produce image signals, where [0164] describe that the camera data is used to record positions of a user’s arm and hand); and
outputting activation data spatially correlating the amount of muscle activation of the body part to the location of the body part in the physical space ([0164] describes that the camera and neuromuscular sensor signals are both used in creating a musculoskeletal representation, where both sensor signals are used to determine and position the arm and hand in space, as well as geometries and forces applied by the user’s arm and hand); and
visually presenting, via a display, a graphical user interface including a plurality of user interface objects arranged in the graphical user interface based at least on the activation data ([0191]-[0192] describe that physical object locations in space can be determined by the camera signal, and that a user interaction can be identified using the combination of the signals, such that a set of control actions in the form of virtual controls can be activated in the interface in response to a specifically identified interaction with a particular object).
Claim 18 recites a computing system comprising: a logic processor; and a storage device holding instructions executable by the logic processor ([0197]) that carry out the method of Claim 1, and is similarly rejected.
With regard to Claim 3, Osborn teaches the method of Claim 1, further comprising:
receiving, from a plurality of muscle activation sensors associated with a same user, a plurality of muscle activation signals corresponding to a plurality of muscles associated with a plurality of body parts of the same user, each muscle activation signal indicating an amount of muscle activation of a corresponding muscle of the plurality of muscles ([0171] describes that a plurality of neuromuscular sensors sense and record neuromuscular signals generated by muscle activity at the wrist or forearm. [0109] describes neuromuscular signals associated with a plurality of body part, including an arm and hand);
receiving, from one or more spatial sensors, one or more spatial signals indicating locations of the plurality of body parts in a physical space ([0164] describe that the camera also captures objects including the body parts of the user represented in the musculoskeletal representation); and
wherein the activation data spatially correlates the amount of muscle activation of each of the plurality of body parts to each of the locations of the plurality of body parts in the physical space ([0164] describes that the camera and neuromuscular sensor signals are both used in creating a musculoskeletal representation, where both sensor signals are used to determine and position the arm and hand in space, as well as geometries and forces applied by the user’s arm and hand).
Claim 20 recites a computing system comprising: a logic processor; and a storage device holding instructions executable by the logic processor ([0197]) that carry out the method of Claim 3, and is similarly rejected.
With regard to Claim 6, Osborn teaches that a location of a user interface object of the plurality of user interface objects in the graphical user interface is set based at least on the activation data. [0187]-[0188] describe that when a user interaction activates virtual controls, the enabled controls can appear at the object with which the user interacts, such as transforming an appearance of the object and enabling mode selection therefor.
With regard to Claim 8, Osborn teaches that the graphical user interface is a three-dimensional, 3D, graphical user interface, wherein the plurality of user interface objects each have a 3D location in the 3D graphical user interface, and wherein the location of the body part is mapped to a 3D location in the 3D graphical user interface. [0191]-[0192] describe that the environment in which the objects and musculoskeletal representation exist is an AR environment, where [0195] describes that the environment is modeled as a 3D environment.
With regard to Claim 12, Osborn teaches that the muscle activation sensor includes an electromyography, EMG, sensor. [0129] includes EMG sensors as types of neuromuscular sensors used in the system.
With regard to Claim 13, Osborn teaches that the spatial sensor includes a camera. [0164] describes the use of a camera in reconstructing positions, geometry, and forces of a user’s musculoskeletal representation of the user’s body parts.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2, 4, 7, 15-17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Osborn, in view of Giurgica-Tiron, et al., U.S. PGPUB No. 2019/0228591 (“G-T”).
With regard to Claim 2, G-T teaches that the activation data is output as a heat map data structure indicating different amounts of muscle activation of the body part at different locations in the physical space. [0061] describes that determined position and force information for a musculoskeletal representation can be represented by a visual representation that uses color to denote the amount of force.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine G-T with Osborn. One of skill in the art would have sought the combination, to improve user experience by enabling a user to visualize how the system is interpreting various actions within the 3D environment.
Claim 19 recites a computing system comprising: a logic processor; and a storage device holding instructions executable by the logic processor ([0197]) that carry out the method of Claim 2, and is similarly rejected.
With regard to Claim 4, G-T teaches a method, further comprising:
receiving, from a plurality of muscle activation sensors associated with a plurality of different users, a plurality of muscle activation signals corresponding to a plurality of a same muscle associated with a same body part of the plurality of different users, each muscle activation signal indicating an amount of muscle activation of a corresponding muscle of the plurality of the same muscles ([0079]-[0085] describes that a statistical model can be generated using recorded signals from multiple users, where recorded signals correspond to multiple sensor signals recording multiple types of movements. [0051] describes that camera signals can likewise be used in modeling);
receiving, from one or more spatial sensors, a plurality of spatial signals indicating locations of the plurality of body parts of the plurality of different users ([0051] describes that camera signals can likewise be used in modeling); and wherein
the activation data spatially correlates the amount of muscle activation of each of the plurality body parts of the plurality of different users to each of the locations of the plurality of body parts ([0079]-[0084] describe generating the model using the plurality of sensor signals, where [0051] describes that the sensors include the camera).
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine G-T with Osborn. One of skill in the art would have sought the combination, to improve system functioning by basing interaction determinations on a large number of interactions, thereby increasing accuracy of determinations in at least some instances.
Claim 15 recites a method including aspects of the method of Claim 5, and the claim is similarly rejected.
With regard to Claim 7, G-T teaches that the graphical user interface is a two-dimensional, 2D, graphical user interface, wherein the plurality of user interface objects each have a 2D location in the 2D graphical user interface, and wherein the location of the body part is mapped to a 2D location in the 2D graphical user interface. [0063] describes display and manipulation of 2D visual elements. It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine G-T with Osborn, to improve system functioning by enabling its use with additional types of display elements.
With regard to Claim 16, G-T teaches that the activation data is output as a heat map data structure indicating different amounts of muscle activation of the body part at different locations in the physical space. [0061] describes that determined position and force information for a musculoskeletal representation can be represented by a visual representation that uses color to denote the amount of force.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine G-T with Osborn. One of skill in the art would have sought the combination, to improve user experience by enabling a user to visualize how the system is interpreting various actions within the 3D environment.
With regard to Claim 17, Osborn teaches that one or more of a size and a location of a user interface object of the plurality of user interface objects is set based at least on the activation data. [0187]-[0188] describe that when a user interaction activates virtual controls, the enabled controls can appear at the object with which the user interacts, such as transforming an appearance of the object and enabling mode selection therefor, thereby basing the location on the activation data.
Claims 5, 9-11 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Osborn, in view of Lu, et al., U.S. PGPUB No. 2020/0133450 (“Lu”).
With regard to Claim 5, Lu teaches that a size of a user interface object of the plurality of user interface objects is set based at least on the activation data. [0028]-[0029] describe that an ergonomic tracker can identify that a user exerted force is over a threshold as indicating strain on a user, and in response, change a size of an element in an interface.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to modify Osborn, to enable identifying excessive user force exerted by muscles during interactions, and to modify interfaces in response. One of skill in the art would have sought the modification, to improve user experience by enabling for reduction in user strain by adapting user interfaces in ways which require less force for interactions.
With regard to Claim 9, Lu teaches tracking interaction of the body part with the plurality of user interface objects; wherein a more-frequently-used user interface object of the plurality of user interface objects having a higher interaction frequency with the body part over the period of time is positioned in the graphical user interface at a location correlated with a smaller amount of muscle activation based at least on the activation data, and wherein a less-frequently-used user interface object of the plurality of user interface objects having a lower interaction frequency over the period of time is positioned in the graphical user interface at a location correlated with a larger amount of muscle activation based at least on the activation data.
[0040]-[0041] describe that locations of an element can be moved when a frequency of interactions is exceeded, and that repeated tasks can prompt the creation of a shortcut user interface element, which presents information closer to a user’s point of visual focus, to allow a user to reach needed interface elements without additional strain, such as caused by performing drags, scrolls, clicks, and other interactions.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to modify Osborn, to enable modifying a user interface in response to detecting excess muscle use or force. One of skill in the art would have sought the modification, to improve user experience by enabling for reduction in user strain by adapting user interfaces in ways which require less force or muscle use for interactions.
With regard to Claim 10, Lu teaches dynamically adjusting a default arrangement of the plurality of user interface objects to a customized arrangement of the plurality of user interface objects in a graphical user interface based at least on the activation data; and visually presenting, via the display, the graphical user interface including the customized arrangement of a plurality of user interface objects.
Lu at [0040]-[0041] describe that locations of an element can be moved when a frequency of interactions is exceeded, or that a size or color can be adjusted, in response to detecting strain, including strain detected from muscle data such as neck strain or squinting. Therefore, colors, sizes, and locations are changeable from an initial, default state.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to modify Osborn, to enable modifying a user interface using data associated with a user’s use of muscles. One of skill in the art would have sought the modification, to improve user experience by enabling for reduction in user strain by adapting user interfaces in ways which require less force or muscle use for interactions.
With regard to Claim 11, Lu teaches that one or more of a size of a user interface object and location of the user interface object in the graphical user interface is dynamically adjusted based at least on the activation data. [0028]-[0029] describe that an ergonomic tracker can identify that a user exerted force is over a threshold as indicating strain on a user, and in response, change a size of an element in an interface.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to modify Osborn, to enable identifying excessive user force exerted by muscles during interactions, and to modify interfaces in response. One of skill in the art would have sought the modification, to improve user experience by enabling for reduction in user strain by adapting user interfaces in ways which require less force for interactions.
With regard to Claim 14, Lu teaches that the spatial sensor includes a touch sensor of a touch-sensitive display device. [0024] describes a touch screen. One of skill in the art would have sought to combine Lu with Osborn, to provide additional familiar input devices with which users can interact with systems, thereby improving user experience with traditional, familiar types of input.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEITH D BLOOMQUIST whose telephone number is (571)270-7718. The examiner can normally be reached M-F, 8:30-5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEITH D BLOOMQUIST/Primary Examiner, Art Unit 2171
6/1/2026
Prosecution Insights