DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Received 04/23/2026
Claim(s) 1-5, 7, 12, 13, 15-20, 22, 27, 28, 30, and 33-36 is/are pending.
Claim(s) 1-4, 7, 12, 13, 15-19, 22, 27, 28, and 30 has/have been amended.
Claim(s) 6, 8-11, 14, 21, 23-26, 29, 31, and 32 has/have been cancelled.
Claim(s) 33-36 has/have been added.
The 35 U.S.C § 103 rejection to claim(s) 1, 2, 5, 7, 12, 13, 16, 17, 20, 22, 27, and 28 have been fully considered in view of the amendments received on 04/23/2026 and are fully addressed in the prior art rejection below.
Response to Arguments
Received 04/23/2026
Regarding independent claim(s) 1 and 16:
Applicant’s arguments (Remarks, Page 8: ¶ 5 to Page 9: ¶ 1), filed 04/23/2026, with respect to the rejection(s) of claim(s) 1 and 16 under 35 U.S.C § 103 have been fully considered and are persuasive. Wherein, Osotio et al. (US PGPUB No. 20190096105 A1) fails to disclose a determined physical interaction performed with the reference object. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of Osotio et al., in view of Osotio et al., in view of Ishikawa et al. (US PGPUB No. 20220237769 A1), and further in view of Baxter et al. (US Patent No. 9541996 B1).
Regarding dependent claims 2-7, 12, 13, 15, 17-22, 27, 28, and 30-32:
Applicant’s arguments (Remarks, Page 9, ¶ 2), filed 04/23/2026, with respect to the rejection(s) of claim(s) 2-7, 12, 13, 15, 17-22, 27, 28, and 30-32 under 35 U.S.C § 103 have been fully considered, due the dependency upon claims 1 and 16 respectively. Wherein, the arguments are not persuasive, regarding reasons as addressed above.
Applicant's arguments filed 04/23/2026 have been fully considered but they are not persuasive; as expressed below.
Regarding independent claim(s) 1 and 16:
Applicant argues (Remarks, Page 8, ¶ 6), that “The Office Action alleges that a relevancy score based on a determined user interaction”.
The Examiner disagrees. Applicant’s arguments fail to view the applied prior art teachings, wherein user’s activities are monitored in relation with (1) identifying that the user has been engaged (i.e. interaction 1) with the couch 102 (Osotio; [¶ 0044]), (2) a chair 202 is introduced to the environment (Osotio; [¶ 0045]) based on the engagement (i.e. interaction 1) of the couch 102 (Osotio; [¶ 0044]), (3) the user focuses (i.e. interaction 2) on the chair 202 (Osotio; [¶ 0046]), (4) the user further expresses interest by gazing (i.e. interaction 3, being an enhancement of interaction 2) further on the chair 202 (Osotio; [¶ 0047]), and (5) the user expresses continued interest (i.e. interaction 4, being an enhancement of interaction 3) in the chair 202 (Osotio; [¶ 0048]). Alternatively, the user can ignore (i.e. interaction 5) the chair 202 (Osotio; [¶ 0049]). Thus, the duration or level of a user’s gaze focus corresponds to determined/calculated relevancy score. In other words, no user gaze (on the chair) implicitly corresponds to a no relevancy score (e.g. score of 0), a user focus (e.g. 5 Sec.) on the chair implicitly corresponds to a low relevancy score (e.g. score of 1), the user continues to gaze (e.g. 10 Sec.) on the chair implicitly corresponds to a medium relevancy score (e.g. score of 2), the user continues to gaze (e.g. 20 Sec.) on the chair implicitly corresponds to a high relevancy score (e.g. score of 3), and the user further continues to gaze (e.g. 40 Sec.) on the chair implicitly corresponds to a very high relevancy score (e.g. score of 4).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claim Objections
Claim 16 is objected to because of the following informalities:
Within line 10, “… the reference object based;” appears to include the word “based” as by mistake. The Examiner will interoperate the issue as follows “… the reference object
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5, 7, 12, 13, 16, 17, 20, 22, 27, and 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Osotio et al., US PGPUB No. 20190096105 A1, hereinafter Osotio, in view of Ishikawa et al., US PGPUB No. 20220237769 A1, hereinafter Ishikawa, and further in view of Baxter et al., US Patent No. 9541996 B1, hereinafter Baxter.
Regarding claim 16, Osotio discloses a system (Osotio; system [¶ 0146-0148], as illustrated within Fig. 11; additionally, bot architecture [¶ 0051], as illustrated within Fig. 5) comprising:
communications circuitry configured to access an electronic device (Osotio; the system [as addressed above] comprises communications circuitry configured to access an electronic device (i.e., hardware, sensor(s), UI, and/or display) [¶ 0146-0148], as illustrated within Fig. 11; moreover, AR/VR headset [¶ 0035]); and
control circuitry (Osotio; the system [as addressed above] comprises control circuitry (i.e. processor) [¶ 0146-0148], as illustrated within Fig. 11) configured to:
receive an indication of an initial object being displayed on a display of the electronic device (Osotio; control circuitry (i.e. processor) [as addressed above] (is) configured to receive an indication of an initial object [¶ 0067-068 and ¶ 0080-0081] being displayed on a display of the electronic device [¶ 0035-0036], as illustrated within Fig. 1);
determine a set of characteristics of the initial object (Osotio; control circuitry (i.e. processor) [as addressed above] (is) configured to determine a set of characteristics of the initial object [¶ 0079-0081 and ¶ 0085-0086]; wherein, contextual information drives operation (of the bot service) [¶ 0060 and ¶ 0068-0077] for determining one or more characteristics of an object [¶ 0080-0081]; additionally, the properties of the object also correspond to a set of characteristics [¶ 0036]);
for each of a plurality of reference objects, calculate a relevancy score (Osotio; control circuitry (i.e. processor) [as addressed above] (is) configured to calculate a relevancy score [¶ 0120-0122 and ¶ 0124] for each of a plurality of reference objects (i.e. objects associated with bots) [¶ 0082, ¶ 0126-0127, and ¶ 0131]; wherein, one or more reference objects are associated with bots [¶ 0020 and ¶ 0079-0081]) based on:
a degree to which the set of characteristics of the initial object is exhibited by the reference object (Osotio; calculate a relevancy score [as addressed above] based on a degree to which the set of characteristics of the initial object is exhibited by the reference object [¶ 0120-0121, ¶ 0124, and ¶ 0127-0129]); and
a determined user interaction with the reference object based (Osotio; calculate a relevancy score [as addressed above] based on a determined user interaction with the reference object (based) [¶ 0035-0044 and ¶ 0121]; additionally, rendering fidelity based on interest of a user [¶ 0093-0097 and ¶ 0099], as illustrated within Fig. 7; wherein, interest of a user is monitored [¶ 0045-0047]);
select, from the plurality of reference objects, a selected reference object based on the calculated relevancy score for the selected reference object (Osotio; control circuitry (i.e. processor) [as addressed above] (is) configured to select, from the plurality of reference objects, a selected reference object based on the calculated relevancy score for the selected reference object [¶ 0124, ¶ 0126-0127, and ¶ 0131]); and
display the selected reference object on the display of the electronic device (Osotio; control circuitry (i.e. processor) [as addressed above] (is) configured to display the selected reference object implicitly on the display of the electronic device [¶ 0045, ¶ 0084-0086, and ¶ 0090-0091], as illustrated within Figs. 1-4; such that, the selection of the reference object is associated with determining one or more bots [as addressed above] wherein, augmented content is injected/placed into the AR/VR environment [¶ 0054 and ¶ 0063]).
Osotio fails to explicitly disclose a plurality of reference objects that are distinct from the initial object;
a determined physical interaction performed with the reference object based;
a selected reference objects that are distinct from the initial object; and
display the selected reference object and an indication of the determined physical interaction performed with the reference object on the display of the electronic device.
However, Ishikawa teaches for each of a plurality of reference objects, calculate a relevancy score (Ishikawa; calculating a relevancy score (i.e. calculated evaluation) for each of a plurality of reference objects (i.e. physical object) [¶ 0083-0085]; moreover, relevancy score further corresponding to a matching factor [¶ 0044, ¶ 0062, and ¶ 0065]; additionally, preformed using a controller [¶ 0041]) based on:
a degree to which the set of characteristics of the initial object is exhibited by the reference object (Ishikawa; calculating a relevancy score (i.e. calculated evaluation) for each of a plurality of reference/physical objects [as addressed above] based on a degree to which the set of characteristics of the initial object is exhibited by the reference object [¶ 0084]; moreover, evaluating a difference in the degree of deformation of a physical object [¶ 0047-0048]; wherein, degree of deformation, is regarding determining characteristics of/to the physical object [¶ 0032, ¶ 0062-0063, and ¶ 0065-0067]); and
a determined physical interaction performed with the reference object based (Ishikawa; calculating a relevancy score (i.e. calculated evaluation) for each of a plurality of reference/physical objects [as addressed above] based on a determined physical interaction performed with the reference object based [¶ 0082-0083 and ¶ 0085]);
select, from the plurality of reference objects, a selected reference object based on the calculated score for the selected reference object (Ishikawa; select a selected reference object based on a calculated/determined score/match for the selected reference object from the plurality of reference objects [¶ 0062-0063 and ¶ 0065]; moreover, reference objects are in relation with a scene [¶ 0032] and/or storage data [¶ 0038-0040]); and
display the selected reference object and an indication of the determined physical interaction performed with the reference object on the display of the electronic device (Ishikawa; display the reference/physical object and an indication of the determined physical interaction performed with the reference object on the display of the electronic device [¶ 0085], as illustrated within Fig. 11; additionally, expressing an arrangement between two objects [¶ 0096-0098], such that the relationship between the two objects can reflect an interaction and/or shared environment [¶ 0099-0100 and ¶ 0102], as illustrated within Fig. 14; wherein, a specified arrangement of objects satisfies physical conditions between the two objects expressing an interaction [¶ 0103]).
Osotio and Ishikawa are considered to be analogous art because both pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce an augmented reality effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Osotio, to incorporate for each of a plurality of reference objects, calculate a relevancy score based on: (i) a degree to which the set of characteristics of the initial object is exhibited by the reference object; and (ii) a determined physical interaction performed with the reference object based; select, from the plurality of reference objects, a selected reference object based on the calculated score for the selected reference object; and display the selected reference object and an indication of the determined physical interaction performed with the reference object on the display of the electronic device (as taught by Ishikawa), in order to provide an improved augmented reality environments that merges virtual and real content in a more realist manner (Ishikawa; [¶ 0004-0006]).
Osotio as modified by Ishikawa fails to disclose select, from the plurality of reference objects, a selected reference object based on the calculated relevancy score for the selected reference object.
However, Baxter teaches select, from the plurality of reference objects, a selected reference object based on the calculated relevancy score for the selected reference object (Baxter; selecting/determining a selected reference object [Col. 12, line 26 to Col. 13, line 2] based on the calculated relevancy score for the selected reference object from the plurality of reference objects [Col. 16, line 34 to Col. 17, line 10, Col. 18, lines 28-41, and Col. 19, lines 12-45]; moreover, similarity measurement associated with shape, size, and/or dimension [Col. 17, line 29 to Col. 18, line 9]); and
display the selected reference object on the display of the electronic device (Baxter; display the selected reference object on the display of the electronic device [Col. 18, line 49 to Col. 19, line 26]).
Osotio in view of Ishikawa and Baxter are considered to be known methods to yield predictable results.
Since all the claimed elements would continue to operate in the same manner, specifically the augmented environment would still be utilized to present visualized information to a user and/or enabling users to interact with aspects of the augmented environment (taught by Osotio in view of Ishikawa) and the object recognition would still provide information and determinations according to one or more features of real physical objects (taught by Baxter). Therefore, the results would be predictable to one of ordinary skill in the art.
As such, it would have been obvious to one of ordinary skill in the art to modify the processing information and object recognition of Osotio as modified by Ishikawa with selection of a reference object based on the calculated relevancy score of Baxter as being no more "than the predictable use of prior-art elements according to their established functions."
Regarding claim 17, Osotio in view of Ishikawa and Baxter further discloses the system of claim 16, wherein the reference object is depicted in a stored image captured using the electronic device that is associated with a user (Ishikawa; the reference object is depicted in a stored image captured using the electronic device that is associated with a user [¶ 0038-0039]; wherein, imaging involves recognition techniques [¶ 0043-0044]; moreover, displaying data is associated with live-action imaging [¶ 0030-0031]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Osotio as modified by Ishikawa and Baxter, to incorporate the reference object is depicted in a stored image captured using the electronic device that is associated with a user (as taught by Ishikawa), in order to provide an improved augmented reality environments that merges virtual and real content in a more realist manner (Ishikawa; [¶ 0004-0006]).
Regarding claim 20, Osotio in view of Ishikawa and Baxter further discloses the system of claim 16, wherein the initial object is a virtual object (Osotio; the initial object [as addressed within the parent claim(s)] is a virtual object (i.e. AR/VR content) [¶ 0019 and ¶ 0035-0036]).
Regarding claim 22, Osotio in view of Ishikawa and Baxter further discloses the system of claim 16, wherein the user interaction includes any one of 1) a real-world object indicated by a gaze of a user (Osotio; the user interaction [as addressed within the parent claim(s)] includes a real-world object indicated by a gaze of a user [¶ 0044-0048 and ¶ 0099]; wherein, a VR object (e.g. chair) is based on a real-world object and real objects (i.e. couch) are within the FOV, are based on a viewing space for which a user interacts with said objects [id. and ¶ 0032-0033]), 2) a virtual object dictated by selection on a website, or 3) a virtual object indicated on a social media feed associated with a user.
Regarding claim 27, Osotio in view of Ishikawa and Baxter further discloses the system of claim 16, wherein the determined physical interaction is a physical interaction performed with the reference object by a contact of a user of the electronic device (Ishikawa; the determined physical interaction is a physical interaction performed with the reference object by a contact of a user of the electronic device [¶ 0080-0084]; moreover, estimated interaction [id.]; wherein, user physical interaction between two objects is determinable [¶ 0047 and ¶ 0085]; additionally, specifying an arrangement of physical objects [¶ 0096-0097], as illustrated within Fig. 13, reflecting an interaction [¶ 0098-0100], as illustrated within Fig. 14; moreover, expressing a capable interaction between two objects [¶ 0102-0103]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Osotio as modified by Ishikawa and Baxter, to incorporate the determined physical interaction is a physical interaction performed with the reference object by a contact of a user of the electronic device (as taught by Ishikawa), in order to provide an improved augmented reality environments that merges virtual and real content in a more realist manner (Ishikawa; [¶ 0004-0006]).
Regarding claim 28, Osotio in view of Ishikawa and Baxter further discloses the system of claim 16, wherein the determined physical interaction is determined, by the control circuitry, based on the reference object being stored in a photo-gallery of the electronic device associated with a user (Ishikawa; the determined physical interaction is determined based on the reference object [¶ 0080-0084] being stored in a photo-gallery of the electronic device associated with a user [¶ 0039-0040 and ¶ 0085] by the control circuitry [¶ 0037-0038 and ¶ 0088]; additionally, an arrangement of physical objects [¶ 0096-0097] in relation with an interaction DB [¶ 0039-0040] and specifying techniques [¶ 0049-0051] in relation with a processing unit [¶ 0054]; moreover, arrangements [as addressed above] based on stored data and using a controller [¶ 0089-0093] in order to reflect interactions [¶ 0102-0103]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Osotio as modified by Ishikawa and Baxter, to incorporate the determined physical interaction is determined, by the control circuitry, based on the reference object being stored in a photo-gallery of the electronic device associated with a user (as taught by Ishikawa), in order to provide an improved augmented reality environments that merges virtual and real content in a more realist manner (Ishikawa; [¶ 0004-0006]).
Regarding claim 35, Osotio in view of Ishikawa and Baxter the system of claim 16, wherein the control circuitry configured to display the selected reference object and the indication of the determined physical interaction performed with the reference object on the display of the electronic device is further configured to simultaneously display the selected reference object and the initial object (Ishikawa; the control circuitry [¶ 0037-0038 and ¶ 0088] configured to display the selected reference object and the indication of the determined physical interaction performed with the reference object on the display of the electronic device [as addressed within the parent claim(s)] is further configured to simultaneously display the selected reference object and the initial object [¶ 0032 and ¶ 0085], as illustrated within Fig. 2 and Fig. 11; moreover, visually arranging a scene in an implicitly overlapping manner [¶ 0099-0100 and ¶ 0102-0103], as illustrated within Fig. 14; wherein, arrangements of a physical interaction are configured by a controller [¶ 0089-0093]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Osotio as modified by Ishikawa and Baxter, to incorporate the control circuitry configured to display the selected reference object and the indication of the determined physical interaction performed with the reference object on the display of the electronic device is further configured to simultaneously display the selected reference object and the initial object (as taught by Ishikawa), in order to provide an improved augmented reality environments that merges virtual and real content in a more realist manner (Ishikawa; [¶ 0004-0006]).
Regarding claim 36, Osotio in view of Ishikawa and Baxter further discloses the system of claim 16, wherein the control circuitry is further configured to determine the determined physical interaction performed with the reference object based at least in part on receiving an indication of an association between the selected reference object and an anticipated physical interaction to be performed with the initial object (Ishikawa; the control circuitry [¶ 0037-0038 and ¶ 0088] is further configured to determine the determined physical interaction performed with the reference object [¶ 0089-0093] based at least in part on receiving an indication of an association between the selected reference object [¶ 0047 and ¶ 0049-0051] and an anticipated physical interaction to be performed with the initial object [¶ 0082-0084]; additionally, determining an interaction is in relation with specifying an arrangement of objects [¶ 0096-0098] which is further associated with an expression configure by a controller [¶ 0099-0100 and ¶ 0102]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Osotio as modified by Ishikawa and Baxter, to incorporate the control circuitry is further configured to determine the determined physical interaction performed with the reference object based at least in part on receiving an indication of an association between the selected reference object and an anticipated physical interaction to be performed with the initial object (as taught by Ishikawa), in order to provide an improved augmented reality environments that merges virtual and real content in a more realist manner (Ishikawa; [¶ 0004-0006]).
Regarding claim 1, the rejection of claim 1 is addressed within the rejection of claim 16, due to the similarities claim 1 and claim 16 share, therefore refer to the rejection of claim 16 regarding the rejection of claim 1.
Regarding claim 2, the rejection of claim 2 is addressed within the rejection of claim 17, due to the similarities claim 2 and claim 17 share, therefore refer to the rejection of claim 17 regarding the rejection of claim 2.
Regarding claim 5, the rejection of claim 5 is addressed within the rejection of claim 20, due to the similarities claim 5 and claim 20 share, therefore refer to the rejection of claim 20 regarding the rejection of claim 5.
Regarding claim 7, the rejection of claim 7 is addressed within the rejection of claim 22, due to the similarities claim 7 and claim 22 share, therefore refer to the rejection of claim 22 regarding the rejection of claim 7.
Regarding claim 12, the rejection of claim 12 is addressed within the rejection of claim 27, due to the similarities claim 12 and claim 27 share, therefore refer to the rejection of claim 27 regarding the rejection of claim 12.
Regarding claim 13, the rejection of claim 13 is addressed within the rejection of claim 28, due to the similarities claim 13 and claim 28 share, therefore refer to the rejection of claim 28 regarding the rejection of claim 13.
Regarding claim 33, the rejection of claim 33 is addressed within the rejection of claim 35, due to the similarities claim 33 and claim 35 share, therefore refer to the rejection of claim 35 regarding the rejection of claim 33.
Regarding claim 34, the rejection of claim 34 is addressed within the rejection of claim 36, due to the similarities claim 34 and claim 36 share, therefore refer to the rejection of claim 36 regarding the rejection of claim 34.
Allowable Subject Matter
Claim(s) 3, 4, 15, 18, 19, and 30 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Prior art to be further consider, regarding the subject matter of proxy objects:
Piya et al. (US Patent No. 11543933 B2);
Thomas et al. (US PGPUB No. 20160189426 A1); and
DeLuca et al. (US PGPUB No. 20180357670 A1).
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of Reference Cited for a listing of analogous art.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Charles Lloyd Beard whose telephone number is (571)272-5735. The examiner can normally be reached Monday - Friday, 8:00 AM - 5: 00 PM, alternate Fridays EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHARLES LLOYD. BEARD
Primary Examiner
Art Unit 2611
/CHARLES L BEARD/ Primary Examiner, Art Unit 2611
Prosecution Insights