Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
1. This is in response to application filed on 02/18/2025 in which claim 14-33 are presented for examination.
Status of Claims
2. Claims 14-33 are pending, of which claim 14, 23 and 32 are in independent form.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 14-17, 22-26 and 31-33 are rejected under 35 U.S.C 103 as being unpatentable over Ma (US 11,113,893) Patented on September 07, 2021 in view of Costa(US PG Pub 2018/0365898) Published on December 20, 2018.
As per claim 14, 23 and 32, Ma teaches (Previously presented) A method comprising:
at a device including an image sensor, a display, one or more processors(Col 3 ln 30-5 Col 4 Ln 45-55 discloses camera and tablet which includes a processor, as taught by Ma), and non- transitory memory:
capturing, using the image sensor, an image of an environment including a plurality of objects(Col 3 ln 30-5 Col 4 Ln 45-55 discloses camera and tablet which includes a processor. tablet with a camera on the back can capture images of the real world, as taught by Ma);
displaying a plurality of group user interface elements in respective association with the plurality of objects, each group user interface element representing a plurality of user interface elements(Fig 7 and 9 Col 18 Ln 10-30 [abstract] Col 15 Ln 35-65 Col 16 Ln 1-40 Glints associated with real world object. the number of glints are grouped and replaced with a group indicator. In some implementations, the group indicator can be another glint, e.g., with an indicator identifying a number of glints in the group it replaced, as taught by Ma);
detecting user input directed to a particular group user interface element of the plurality of group user interface elements associated with a particular object of the plurality of objects(col 16 Ln 1-50 Col 17 Ln 1-50 discloses the user directs a gesture (e.g., an “air click”) or otherwise selects the group indicator, as taught by Ma) ; and
Ma teaches various glints can be selected and displayed. glints include an icon representation the application that the glint links to and an image relating to content from the application associated with the object the glint is associated with(Col 17 Ln 60-65 Col 18 Ln 1-20). However MA does not explicitly teach
detecting, in the image of the environment, a plurality of actionable items respectively associated with a plurality of actions;
relating each of the plurality of actionable items to a respective one of the plurality of objects;
displaying, on the display, a plurality of user interface elements in respective association with those of the plurality of actionable items related to the particular object.
On the other hand, Costa teaches detecting, in the image of the environment, a plurality of actionable items respectively associated with a plurality of actions(Fig 7 -10 and 13 Para[0071-0074][0082] discloses multiple selectable elements corresponding to real world object , as taught by Costa);
relating each of the plurality of actionable items to a respective one of the plurality of objects(Fig 7 -10 and 13 Para[0071-0074][0082] discloses multiple selectable elements corresponding to real world object e.g.1210, 728-750, as taught by Costa);
displaying, on the display, a plurality of user interface elements in respective association with those of the plurality of actionable items related to the particular object(Fig 7 -10 and 13 Para[0071-0074][0082] discloses multiple selectable elements corresponding to real world object. Knob 1210 can be used to adjust (e.g. increase the volume of the music player or decrease the volume of the music player), as taught by Costa).
It would have been obvious to one of ordinary skill in the art before the filing date of the
invention to modify Ma invention with the teaching of Costa because doing so would
result in efficiency by executing the first software application on the mixed reality device in association
with the first real-world object.
As per claim 15 and 24, the combination of Ma and Costa teaches wherein detecting the
plurality of actionable items includes detecting machine-readable content(fig 13 Col 20 Ln 1-50 e.g.
1305, as taught by Ma).
As per claim 16 and 25, the combination of Ma and Costa teaches wherein detecting the
plurality of actionable items includes detecting an object of the plurality of objects(fig 13 Col 20 Ln 1-50 a
first object 1302 can include a QR code 1305 included on the object, as taught by Ma).
As per claim 17, 26 and 33, the combination of Ma and Costa teaches further comprising:
detecting user input directed to a particular user interface element of the plurality of user interface elements associated with a particular actionable item of the plurality of actionable items(Para[0082] user interacts with knob 1210 by turning or rotating their fingers in a first direction, corresponding to the real world object, as taught by Costa); and
performing an action associated with the particular actionable item(Para[0082] user interacts with knob 1210 by turning or rotating their fingers in a first direction to increase or decrease the volume, corresponding to the real world object, as taught by Costa).
As per claim 22 and 31, the combination of Ma and Costa teaches wherein each of the plurality of user interface elements is displayed differently than each of the plurality of group user interface elements(Col 3-4 Ln 45-50 and 1-15, as taught by Ma).
6. Claims 18-20 and 27-29 are rejected under 35 U.S.C 103 as being unpatentable over Ma (US 11,113,893) Patented on September 07, 2021 in view of Costa(US PG Pub 2018/0365898) Published on December 20, 2018 in further view of Zhou(US PG Pub 2014/0330656) published on November 06, 2014.
As per claim 18 and 27, the combination of Ma and Costa Does not explicitly teach wherein
performing the action associated with the particular actionable item includes displaying content relating to
the particular actionable item.
On the other hand, Zhou teaches wherein performing the action associated with the particular
actionable item includes displaying content relating to the particular actionable item(Para[0010] displayed barcode may be scanned by a barcode scanner of the merchant and processed to retrieve data of the mobile payment account. Based on this data, a payment request may be generated and sent to the mobile payment service provider, as taught by Zhou).
It would have been obvious to one of ordinary skill in the art before the filing date of the
invention to modify Ma and Costa invention with the teaching of Zhou because doing so would
result in efficiency by allowing the easily received information embedded in the bar code by scanning the
bar code.
As per claim 19 and 28, the combination of Ma, Costa and Zhou teaches wherein performing the
action associated with the particular actionable item includes changing a state of the particular actionable
item(Fig 7 -10 and 13 Para[0071-0074][0082] discloses multiple selectable elements corresponding
to real world object. Knob 1210 can be used to adjust (e.g., increase the volume of the music
player or decrease the volume of the music player), as taught by Costa).
As per claim 20 and 29, the combination of Ma, Costa and Zhou teaches wherein relating each of the plurality of actionable items to a respective one of the plurality of object includes performing image segmentation on the image of the environment(Para[0032][0051][0081-0086], as taught by Costa).
7. Claims 21 and 30 are rejected under 35 U.S.C 103 as being unpatentable over Ma (US 11,113,893) Patented on September 07, 2021 in view of Costa(US PG Pub 2018/0365898) Published on December 20, 2018 in further view of Dascola(US PG Pub 2019/0342251) published on November 07, 2019.
As per claim 21 and 30, the combination of Ma and Costa Does not explicitly teach further comprising:
displaying a super-group user interface element in association with an object including two or more of the plurality of objects, the super-group user interface element representing the plurality of group user interface elements; and
detecting user input directed to the super-group user interface element, wherein displaying the plurality of group user interface elements is performed in response to detecting the user input directed to the super-group user interface element.
On the further comprising:
displaying a super-group user interface element in association with an object including two or more of the plurality of objects, the super-group user interface element representing the plurality of group user interface elements(Para[0212-0214][0255-0256] shows multiple object e.g. 5206, 5196, 5180, and 5068 coalesced representation of notifications message group, as taught by Dascola); and
detecting user input directed to the super-group user interface element, wherein displaying the plurality of group user interface elements is performed in response to detecting the user input directed to the super-group user interface element(Para[0212-0214][0255] shows multiple object e.g. 5206, 5196, 5180, and 5068 coalesced representation of notifications message group. a tap input by contact 5222 is detected at a location that corresponds to control 5220 for re-coalescing the expanded version of the set of notifications. In response to the input, the coalesced representation of notifications that includes notifications 5066, 5068, 5180, 5196, and 5206 is redisplayed, as taught by Dascola).
It would have been obvious to one of ordinary skill in the art before the filing date of the
invention to modify Ma and Costa invention with the teaching of Dascola because doing so would
result in increased visibility by allowing the user to access the notification without requiring input for
navigating away from the wake screen user interface.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYEEZ R CHOWDHURY whose telephone number is (571)270-3069. The examiner can normally be reached Monday-Friday 9AM-6:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William L Bashore can be reached at 571-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYEEZ R CHOWDHURY/Primary Examiner, Art Unit 2174 Thursday, April 2, 2026