DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments filed 1/26/2026, with respect to claims 1-20 have been fully considered but are moot in view new ground(s) of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8, 10, 13, 14, 107-111, 113 and 114 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kahan et al. (PGPUB Document No. US 2023/0237192) in view of Hosangadi et al. (PGPUB Document No. US 2024/0064266) in view of Ayush et al. (PGPUB Document No. US 2019/0378204).
Regarding claim 1, Kahan teaches a method comprising:
Receiving a first image of a real-world environment around a first device, wherein (An augmented reality environment may involve live direct or indirect view of a physical real-world environment that is around the user devices 1802, 1806 (Kahan: 0062, 0295, FIG.18)):
The image comprises a first object and a second object (first and second views 1804, 1808 comprising a plurality of objects such as virtual objects 1702, 1704, 1706 (Kahan: 0290, 0296, 0303, FIG.18, FIG.17));
And the first object and second object are depictions of real-world objects (virtual objects shown in FIG.17 are depictions of real-world objects such as a sun/cloud, plant, envelope, and Rubik’s cube. Further, Kahan teaches classifying portions of the physical environment as private or public using a machine learning model trained using training examples (Kahan: 0407));
Extracting a first plurality of features from the first image, wherein the first plurality of features correspond to a first object of the plurality of objects (the object recognition process comprising extracting image features such as edges and corners (Kahan: 0520, 0517, 0340, 0127) and the recognition process based on the machine learning model (Kahan: 0407))
Extracting a second plurality of features from the first image, wherein the second plurality of features correspond to the second object (the object recognition process of Kahan being applied to any other object corresponds to the second object):
Determining that the first object is a first type of object, wherein the first type of object corresponds to a private object (classifying virtual objects 1706, 1704 as private objects (Kahan: 0296, 0316) based on the object independent of location (Kahan: 0278), or classifying certain objects/portions within the physical environment as private objects (Kahan: 0409));
Transforming the first type of object into a transformed first type of object (“the first virtual object may be a replica of a child and the distorted representation may obscure the face of the child but allow viewing non-facial features of the child” (Kahan: 0328). “For instance, distorted presentation 2300 may include hashed markings to obfuscate first virtual object 1706” (Kahan: 0329));
And transmitting the transformed feature set to a device (implementing the teachings of Kahan at a remote processing unit 208 requires data to be transmitted between the remote processing unit 208 and XR unit 204).
Receiving one or more actions relating to the transformed feature set (user initiated movement of the virtual object (Kahan: 0309) within an XR environment (Kahan: 0073));
And performing the one or more actions on a first virtual object in an XR environment, wherein the first virtual object corresponds to the first object (the resulting movement of the virtual object).
Note, Kahan contains a “base” process of applying masks to private objects in an XR environment. However, Kahan is silent as to whether masking process comprise the step of “transforming the first object type comprising, the first plurality of features into a transformed feature set using a keyed feature transform algorithm corresponding to a transform key” as presently claimed.
Hosangadi teaches a known technique of “transforming the first object type comprising, the first plurality of features into a transformed feature set using a keyed feature transform algorithm corresponding to a transform key” that is applicable to the “base process (encoding the one or more feature vectors via the process 411 into at least one unencrypted mask that obscures the at least one object in the one or more images (Hosangadi: 0087, 0086, FIG.4)).
Hosangadi’s known technique would have been recognized by one skill in the art as applicable to the “base” process of Kahan and the results would have been predictable and resulted in effectively applying a mask to objects detected in the image/video captured by the XR device of Kahan, which results in an improved process.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Further, the combined teachings above do not expressly teach but Ayush teaches
Generating a first XR environment comprising a first virtual object and a second virtual object using the first plurality of features and the second plurality of features, wherein: (Ayush teaches the concept identifying a physical object such as the chair 406 (Ayush: FIG.4) and overlaying a corresponding AR chair 702 (Ayush: FIG.7A, 0109). Applying the teachings Ayush enables identifying physical objects within the physical environment of Kahan (Kahan: 0407) and overlaying corresponding AR objects as taught by Ayush)
The first virtual object corresponds to the first object depicted in the image (the AR chair 702 corresponds to real world chair 406 (Ayush: 0109, FIG.4, FIG.7A)) and is generated using the first plurality of features (identifying a first real world object using either the teachings of Kahn (Kahan: 0520, 0517, 0340, 0127, 0407) or Ayush (Ayush: 0024))
And the second virtual object corresponds to the second object depicted in the image and is generated using the second plurality of features (any of the other identified objects such as the bookcase, frames (Ayush: 0070, FIG.4))
Generating the first XR environment for display for the first device (the resulting AR at the user devices 1806 of Kahn that is not approved to display private information (Kahn: 0296));
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to identify physical objects and overlay corresponding AR content as taught by Ayush, because this enables an AR environment with an added variety of AR elements.
The combined teachings above now teach,
Generating a second XR environment comprising the second virtual object and a third virtual object using the transformed feature set and the second plurality of features, wherein the third virtual object corresponds to the first object depicted in the image and is generated using the transformed features set (the AR environment of approved device 1802 of Kahn (Kahn: 0296, FIG.18), comprising second and third virtual objects superimposed over real world object according to the teachings of Ayush as stated in above, wherein the third virtual object corresponds to an object identified as being private (not masked). Hence, the virtual object is displayed without being masked)
Generating the second XR environment for display for a second device (the resulting AR displayed at the approved device 1802 of Kahn)
Receiving a first action relating to the transformed feature set, wherein the first action correspond to a change of the third virtual object in the second XR environment (moving the object classified as private of Kahn to a location that changes the classification to being non-private (Khan teaches the concept of moving objects between locations that classify objects as either private or non-private, 0310));
And performing a second action on the first virtual object in the first XR environment, wherein the second action is based, at least in part, on the first action (the action of moving the first object within the first AR environment of user 1806 of Kahn (Kahn: 0310)).
Regarding claim 2, the combined teachings teach the method of claim 1, further comprising: determining that the second object is a second type of object (classify one or more of virtual objects 1700 to 1706 as public (Kahan: 0296, 0316)).
Regarding claim 3, the combined teachings teach the method of claim 1, wherein the first object is determined to be of the first type of object based on one or more of the first plurality of features (classifying a virtual object as private (Kahan: 0361) and identifying said object utilizing the recognition processing comprise identifying one or more features (Kahan: 0520, 0517, 0340, 0127))
Regarding claim 4, the combined teachings teach the method of claim 1, further comprising encrypting the transformed feature set to create an encrypted dataset, transmitting the encrypted dataset (Kahan teaches the concept of sharing private information such as the private virtual object in an encrypted format (Kahan: 0291)).
Regarding claim 5, the combined teachings teach the method of claim 1, further comprising transmitting metadata indicating that the transformed feature set corresponds to the first type of object (classification information classifying an object as private (Kahan: 0296)).
Regarding claim 6, the combined teachings teach the method of claim 1, further comprising determining, by the second device, whether the transformed feature set corresponds to at least one virtual object of a plurality of virtual objects stored at the second device (retrieving objects from a database associated with privacy levels (Kahan: 0092), wherein said information is stored the first and/or second device).
Regarding claim 8, the combined teachings teach the method of claim 1, wherein transforming the first plurality of features into the transformed feature set is performed by an XR application processing unit (the processor for carrying out the transformation (Kahan: 0517)).
Regarding claim 10, the combined teachings teach the method of claim 1, wherein the first device is a first XR device comprising a first display; and the second device is a second XR device comprising a second display (first and second display of the first and second extended reality appliance 1806, 1802 (Kahn: 0297-0298, FIG.18)).
Regarding claim 13, the combined teachings teach the method of claim 12, wherein the second action corresponds to changing an orientation or position of the virtual object in the first XR environment (moving virtual objects (Kahan: 0311, 0314)).
Regarding claim 14, the combined teachings teach the method of claim 12, wherein the second action corresponds to changing a shape or size of the virtual object in the first XR environment (“tailor, adapt, and/or adjust” parameter settings to display content (Kahan: 0160)).
Regarding claim 113, the combined teachings above teach the method of claim 1, further comprising:
Generating a plurality of updated features related to the third virtual object, wherein receiving the first action relating to the transformed feature set comprises receiving the plurality of updated features (within the AR environment of 1802, moving the object classified as private of Kahn to a location that changes the classification to being non-private (Khan teaches the concept of moving objects between locations that classify objects as either private or non-private, 0310), which removes the mask applied according to the teachings of Hosangadi (updated feature));
Determining that the plurality of updated features corresponds to the first virtual object, wherein performing the second action on the first virtual object in the first XR environment is in response to determining that the plurality of updated features corresponds to the first virtual object (moving the first virtual object within AR environment of 1806 of Kahn also results in being moved within the AR environment of 1802. Displaying based on whether it has moved to a private or non-private location (Kahn: 0310) and further based on whether the device is approved for presentation of private information (Kahn: 0293)).
Regarding claim 114, the combined teachings above teach the method of claim 113, wherein:
Determining that the plurality of updated features corresponds to the first virtual object comprises back transforming the plurality of updated features into an updated first plurality of features (removing the initially applied mask according to Hosangadi based on the newly moved location of the first virtual object);
And the second action is performed on the first virtual object in the first XR environment using the updated first plurality of features (corresponds to the first virtual object no longer masked and visible to the first user 1806 of Kahn while being moved).
Regarding claim 107, Kahan teaches a method comprising:
Receiving a first image of a real-world environment around a first device, wherein (An augmented reality environment may involve live direct or indirect view of a physical real-world environment (Kahan: 0062)):
The image comprises a plurality of objects (first and second views 1804, 1808 comprising a plurality of objects such as virtual objects 1702, 1704, 1706 (Kahan: 0290, 0296, 0303, FIG.18, FIG.17));
And each object of the plurality of objects are depictions of real-world objects (virtual objects shown in FIG.17 are depictions of real-world objects such as a sun/cloud, plant, envelope, and Rubik’s cube);
Generating a plurality of virtual objects, wherein each virtual object of the plurality of virtual objects corresponds to at least one object of the plurality of objects (virtual objects 1704 and 1706 corresponds to the location of the desk 1504, and virtual object 1702 correspond to the location of the side-table 1506 (Kahan: 0312, FIG.18));
Generating an XR environment for display, wherein the XR environment comprises the plurality of virtual objects (the AR view 1804 and 1808 if FIG.18 (Kahan: 0303)),
Extracting a first plurality of features from the first image, wherein the first plurality of features correspond to a first object of the plurality of objects (the object recognition process comprising extracting image features such as edges and corners (Kahan: 0520, 0517, 0340, 0127));
Determining that the first object is a depiction of a first type of real-word object, wherein the first type of real-world object corresponds to a private real-world object (the object recognition process comprising extracting image features such as edges and corners (Kahan: 0520, 0517, 0340, 0127), and classifying virtual objects 1706, 1704 as private objects (Kahan: 0296, 0316) based on the object independent of location (Kahan: 0278));
Transforming the first type of object into a transformed first type of object (“the first virtual object may be a replica of a child and the distorted representation may obscure the face of the child but allow viewing non-facial features of the child” (Kahan: 0328). “For instance, distorted presentation 2300 may include hashed markings to obfuscate first virtual object 1706” (Kahan: 0329));
Transmitting the transformed feature set to a device (implementing the teachings of Kahan at a remote processing unit 208 requires data to be transmitted between the remote processing unit 208 and XR unit 204);
Receiving one or more actions relating to the transformed feature set (user initiated movement of the virtual object (Kahan: 0309) within an XR environment (Kahan: 0073));
And performing the one or more actions on a first virtual object in the XR environment, wherein the first virtual object corresponds to the first object (the resulting movement of the virtual object).
Note, Kahan contains a “base” process of applying masks to private objects in an XR environment. However, Kahan is silent as to whether masking process comprise the step of “transforming the first object type comprising, the first plurality of features into a transformed feature set using a keyed feature transform algorithm corresponding to a transform key” as presently claimed.
Hosangadi teaches a known technique of “transforming the first object type comprising, the first plurality of features into a transformed feature set using a keyed feature transform algorithm corresponding to a transform key” that is applicable to the “base process (encoding the one or more feature vectors via the process 411 into at least one unencrypted mask that obscures the at least one object in the one or more images (Hosangadi: 0087, 0086, FIG.4)).
Hosangadi’s known technique would have been recognized by one skill in the art as applicable to the “base” process of Kahan and the results would have been predictable and resulted in effectively applying a mask to objects detected in the image/video captured by the XR device of Kahan, which results in an improved process.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Further, the combined teachings above do not expressly teach but Ayush teaches each object of the plurality of objects are depictions of real-world objects located in the real-world environment around the first device (Ayush teaches the concept identifying a physical object such as the chair 406 (Ayush: FIG.4) and overlaying a corresponding AR chair 702 (Ayush: FIG.7A, 0109). Applying the teachings Ayush enables identifying physical objects within the physical environment of Kahan (Kahan: 0407) and overlaying corresponding AR objects as taught by Ayush)
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to identify physical objects and overlay corresponding AR content as taught by Ayush, because this enables an AR environment with an added variety of AR elements.
Claims 108-111 is similar in scope to claims 13, 14, 105 and 106.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kahan in view of Hosangadi in view of Ayush as applied to the claim(s) above, and further in view of Jindal et al. (PGPUB Document No. US 2022/0261579).
Regarding claim 7, the combined teachings do not expressly teach but Jindal teaches the method of claim 6, further comprising storing a new virtual object corresponding to the transformed feature set based on determining that the transformed feature set does not correspond to at least one virtual object of a plurality of virtual objects stored at the second device (allows for new object images to be easily added to the database (Jindal: 0028) within an augmented reality environment (Jindal: abstract)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to allow the user to add new virtual content, because this enables an added variety of content to be viewable within the XR environment of the combined teachings above.
Claim(s) 112 and 115 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kahan in view of Hosangadi as applied to the claim(s) above, and further in view of Kim et al. (PGPUB Document No. US 2015/0310671).
Regarding claim 112, Kahan teaches a method comprising:
Receiving a first image of a real-world environment around a first device, wherein (An augmented reality environment may involve live direct or indirect view of a physical real-world environment (Kahan: 0062)):
The image comprises a first object and a second object (first and second views 1804, 1808 comprising a plurality of objects such as virtual objects 1702, 1704, 1706 (Kahan: 0290, 0296, 0303, FIG.18, FIG.17));
And the first object and second object are depictions of real-world objects (virtual objects shown in FIG.17 are depictions of real-world objects such as a sun/cloud, plant, envelope, and Rubik’s cube);
Extracting a first plurality of features from the first image, wherein the first plurality of features correspond to a first object of the plurality of objects (the object recognition process comprising extracting image features such as edges and corners (Kahan: 0520, 0517, 0340, 0127) and the recognition process based on the machine learning model (Kahan: 0407))
Extracting a second plurality of features from the first image, wherein the second plurality of features correspond to the second object (the object recognition process of Kahan being applied to any other object corresponds to the second object):
Generating a first XR environment for display for the first device (the AR environment presented to the user device 1802 (Kahan: 0062, 0295, FIG.18));
Note, Kahan contains a “base” process of applying masks to private objects in an XR environment. However, Kahan is silent as to whether masking process comprise the step of “transforming the first object type comprising, the first plurality of features into a transformed feature set using a keyed feature transform algorithm corresponding to a transform key” as presently claimed.
Hosangadi teaches a known technique of “transforming the first object type comprising, the first plurality of features into a transformed feature set using a keyed feature transform algorithm corresponding to a transform key” that is applicable to the “base process (encoding the one or more feature vectors via the process 411 into at least one unencrypted mask that obscures the at least one object in the one or more images (Hosangadi: 0087, 0086, FIG.4)).
Hosangadi’s known technique would have been recognized by one skill in the art as applicable to the “base” process of Kahan and the results would have been predictable and resulted in effectively applying a mask to objects detected in the image/video captured by the XR device of Kahan, which results in an improved process.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Further, the combined teachings above do not expressly teach but Kim teaches
Generating the first virtual object of the plurality of virtual objects within the XR environment using the first plurality of features, wherein the first virtual object comprises a first set of coordinates within the XR environment (virtual objects associated with coordinates defined within AR space (Kim: 0008));
And changing the first virtual object from the first set of coordinates within the XR environment to a second set of coordinates within the XR environment based, at least in part, on the plurality of updated features (Kim teaches the concept of moving position coordinates of virtual objects within AR space (Kim: 0008)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such that the virtual objects positions are represented by coordinates as taught by Kim, because this enhanced efficiency and accuracy in defining and updating virtual objects.
Further, the combined teachings above do not expressly teach but Ayush teaches
Generating a first XR environment comprising a first virtual object and a second virtual object using the first plurality of features and the second plurality of features, wherein: (Ayush teaches the concept identifying a physical object such as the chair 406 (Ayush: FIG.4) and overlaying a corresponding AR chair 702 (Ayush: FIG.7A, 0109). Applying the teachings Ayush enables identifying physical objects within the physical environment of Kahan (Kahan: 0407) and overlaying corresponding AR objects as taught by Ayush)
The first virtual object corresponds to the first object depicted in the image (the AR chair 702 corresponds to real world chair 406 (Ayush: 0109, FIG.4, FIG.7A)) and is generated using the first plurality of features (identifying a first real world object using either the teachings of Kahn (Kahan: 0520, 0517, 0340, 0127, 0407) or Ayush (Ayush: 0024))
And the second virtual object corresponds to the second object depicted in the image and is generated using the second plurality of features (any of the other identified objects such as the bookcase, frames (Ayush: 0070, FIG.4))
Generating the first XR environment for display for the first device (the resulting AR at the user devices 1806 of Kahn that is not approved to display private information (Kahn: 0296));
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to identify physical objects and overlay corresponding AR content as taught by Ayush, because this enables an AR environment with an added variety of AR elements.
The combined teachings above now teach,
Generating a second XR environment comprising the second virtual object and a third virtual object using the transformed feature set and the second plurality of features, wherein the third virtual object corresponds to the first object depicted in the image and is generated using the transformed features set (the AR environment of approved device 1802 of Kahn (Kahn: 0296, FIG.18), comprising second and third virtual objects superimposed over real world object according to the teachings of Ayush as stated in above, wherein the third virtual object corresponds to an object identified as being private (not masked). Hence, the virtual object is displayed without being masked)
Generating the second XR environment for display for a second device (the resulting AR displayed at the approved device 1802 of Kahn)
Receiving a first action relating to the transformed feature set, wherein the first action correspond to a change of the third virtual object in the second XR environment (moving a the object classified as private of Kahn to a location that changes the classification to being non-private (Khan teaches the concept of moving objects between locations that classify objects as either private or non-private, 0310));
And performing a second action on the first virtual object in the first XR environment, wherein the second action is based, at least in part, on the first action (the action of removing the mask of Kahn once the object is determined to be located in a non-private location (Kahn: 0310)).
Receiving a plurality of updated features related to the third virtual object from the second device, wherein the plurality of updated features correspond to changing a position of the third virtual object in the second XR environment (moving a the object classified as private of Kahn within the second AR environment of device 1802 to a location that changes the classification to being non-private (Khan teaches the concept of moving objects between locations that classify objects as either private or non-private, 0310));
And changing the first virtual object from the first set of coordinates within the first XR environment to a second set of coordinates within the first XR environment based, at least in part, on the plurality of updated features (the resulting movement of the first virtual object (Kahn, 0310) based on the coordinate teachings of Kim stated above (Kim: 0008) within the first AR environment of device 1806 of Kahn).
Regarding claim 115, the combined teachings above teach the method of claim 112, further comprising:
Back transforming the plurality of updated features related to the third virtual object into an updated first plurality of features (removing the mask applied according to the teachings of Hosangadi as stated in the rejection above),
Wherein changing the first virtual object from the first set of coordinates within the first XR environment to the second set of coordinates within the first XR environment is based, at least in part, on updated first plurality of features (masking or removing a mask applied according to the teachings of Hosangadi as stated above, which is determined to be visible or not to the user based on the location that the object has been moved to and/or the approval of the user device according to the teachings of Kahan above).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights