DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Amendment
The amendment filed 06/08/2026 has been entered.
Response to Arguments
Applicant argues Grundhoefer (US 20210097714 A1) does not teach or suggest at least moving an origin of a virtual space according to a movement amount of the marker, and then redisplaying a virtual object at a preset position in a three-dimensional coordinate system based on the moved origin.
Examiner replies Grundhoefer discloses “redisplay the virtual object at the preset position in the three-dimensional coordinate system based on the origin after the origin is moved” ([0092] “In some implementations, visual markers are mounted on objects that can move. For example, visual markers can be mounted on food trucks that repeatedly move, periodically move, or day-by-day move their location. Alternatively, a food truck could move to different locations for breakfast, lunch, and dinner. In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.). In some implementations, each changeable geolocation of the moving visual marker is crowd-sourced. In some implementations, metadata associated with the visual marker or the moving visual marker is maintained at the remote location service (e.g., location service 650).”)
Applicant argues Yoshii (US 20230326096 A1) does not qualify as prior art because the provisions of 35 U.S.C. 102(b)(2)(C) are met.
Examiner replies Applicant’s arguments, see pages 8-9, filed 06/08/2026, with respect to c have been fully considered and are persuasive. The rejection of claims 7 and 8 have been withdrawn.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4 and 11-12 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Grundhoefer et al. (US 20210097714 A1) (Hereinafter Grundhoefer).
Regarding claim 1, Grundhoefer discloses a mixed reality device, configured to: (para. [0034], “Examples of CGR include virtual reality and mixed reality “; para. [0040], “There are many different types of electronic systems that enable a person to sense and/or interact with various CGR environments. Examples include head mounted systems, projection-based systems, heads-up displays (HUDs),”)
set an origin of a virtual space by using a marker located in a real space; (para. [0053], “In some implementations, the location module 340 includes a visual marker detection unit 342, a relative positioning unit 344, and a location determination unit 346. The visual marker detection unit 342 is configured to detect and identify visual markers in images of a physical environment. The relative positioning unit 344 is configured to determine relative positioning to a detected object or visual marker from images of the physical environment. The location determination unit 346 is configured to determine a calculated location of objects given a geolocation (crowd-sourced or signal-based positioning) and determined relative positionings.”)
display a virtual object at a preset position in a three-dimensional coordinate system, the three-dimensional coordinate system being based on the origin; (para. [0032], “In CGR, a subset of a person's physical motions, or representations thereof, are tracked, and, in response, one or more characteristics of one or more virtual objects simulated in the CGR environment are adjusted in a manner that comports with at least one law of physics. For example, a CGR system may detect a person's head turning and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment.”)
acquire a movement amount of the marker with respect to a reference position of the marker, the reference position of the marker being acquired when the origin is set; and (para. [0092], “In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.).
move the origin according to the movement amount when the movement amount is greater than a first threshold, the first threshold being preset; (para. [0092], “In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.).”)
redisplay the virtual object at the preset position in the three-dimensional coordinate system based on the origin after the origin is moved. (para. [0092], “In some implementations, visual markers are mounted on objects that can move. For example, visual markers can be mounted on food trucks that repeatedly move, periodically move, or day-by-day move their location. Alternatively, a food truck could move to different locations for breakfast, lunch, and dinner. In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.). In some implementations, each changeable geolocation of the moving visual marker is crowd-sourced. In some implementations, metadata associated with the visual marker or the moving visual marker is maintained at the remote location service (e.g., location service 650).”)
Regarding claim 2, Grundhoefer discloses the device according to claim 1, further configured to: recognize the marker based on an image of the marker, the origin being set by using a position and an orientation of the marker as a reference. (para. [0061], “In some implementations, the relative positioning determines the pose (e.g., position and orientation) of the detected visual marker 410 relative to the pose of the electronic device 420.”)
Regarding claim 3, Grundhoefer discloses the device according to claim 2, further configured to:
reset the origin when the marker is reimaged after the moving of the origin, the resetting being performed by using a position and an orientation of the marker recognized based on an image captured by the reimaging as a reference. (para. [0061], As shown in FIG. 4 and FIG. 5, the electronic device 420 can determine a relative positioning between the electronic device 420 and the visual marker 410 based on the image or images (e.g., the CGR environments 405A) of the physical environment 405 including the visual marker 410. In some implementations, the relative positioning determines the relative orientation of the visual marker 410 with respect to the electronic device 420.”)
Regarding claim 4, Grundhoefer discloses the device according to claim 1, further configured to: repeatedly acquire the movement amount; and determine whether or not the movement amount is greater than the first threshold each time the movement amount is acquired. (para. [0092], “In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.). In some implementations, each changeable geolocation of the moving visual marker is crowd-sourced.”)
Regarding claim 11, similar reasoning as discussed in claim 1 is applied.
Regarding claim 12, claim 12 is a non- transitory computer readable storage medium configured to store a program (para. [0012], “In accordance with some implementations, a non-transitory computer readable storage medium has stored therein instructions, which, when executed by one or more processors of a device, cause the device to perform or cause performance of any of the methods described herein.”) and similar reasoning as discussed in claim 11 is applied.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Grundhoefer et al. (US 20210097714 A1) (Hereinafter Grundhoefer) in view of Marchand (US 20210158587 A1) (Hereinafter Marchand).
Regarding claim 5, Grundhoefer does not explicitly disclose the device according to claim 1, further configured to: calculate the movement amount by using an image of an article on which a task is performed, or by using a detection result from a sensor fixed with respect to the article.
However, Marchand more explicitly teaches, in the context of mixed reality device, The device according to claim 1, further configured to: calculate the movement amount by using an image of an article on which a task is performed, or by using a detection result from a sensor fixed with respect to the article. (Marchand, para [0014], “In association with either or both of the first and second spatial-positioning subsystems, algorithmic triangulation may be employed with the aid of sensors, transducers, transmitters, and receivers that transmit and/or receive electromagnetic energy including, by way of example, radio frequencies and/or light, such as infrared.”; Marchand, para. [0017], “Virtual-reality and mixed-reality headsets employ various on-board sensors such as, but way of example, an inertial measurement unit (IMU), which may include an accelerometer, gyroscope, and a magnetometer; “environment understanding” sensors; one or more cameras; and one or more light-sensors”; Marchand, para. [0020], “Alternatively or additionally, tool workpiece object identification can be facilitated with camera imagery acquired by one or more cameras carried by the headset or otherwise located to acquire images of objects within the physical space.”)
As both Grundhoefer and Marchand are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the device according to claim 1, further configured to: calculate the movement amount by using an image of an article on which a task is performed, or by using a detection result from a sensor fixed with respect to the article, in the context of mixed reality device, by Grundhoefer according to the teaching of Marchand in order to increase the efficiency of the user and to reduce procedural errors (Marchand, para. [0006]).
Regarding claim 6, Grundhoefer does not explicitly disclose the device according to claim 1, wherein the virtual object is displayed to correspond to a fastening location of an article on which a task is performed.
However, Marchand more explicitly teaches, in the context of mixed reality device, the device according to claim 1, wherein the virtual object is displayed to correspond to a fastening location of an article on which a task is performed (Marchand, 350 in FIG 4B; para. [0054], “illustrative machine-generated indicia 350 are presented to the system user HSO wearing the headset 300 in accordance with a current procedure step 135 as dictated by the machine-executable procedure program 130.).
PNG
media_image1.png
302
434
media_image1.png
Greyscale
As both Grundhoefer and Marchand are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the device according to claim 1, wherein the virtual object is displayed to correspond to a fastening location of an article on which a task is performed, in the context of mixed reality device, by Grundhoefer according to the teaching of Marchand in order to increase the efficiency of the user and to reduce procedural errors (Marchand, para. [0006]).
Allowable Subject Matter
Claim 7 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 7, the cited prior art in combination or alone fail to teach calculate a distance between a prescribed object and the virtual object; and when the distance is less than a second threshold, estimate that the task is being performed at the fastening location, the second threshold being preset, in combination with all the limitation recited in the independent claim and any intervening claims.
Claims 8-10 and 13-17 are allowed.
The following is an examiner’s statement of reasons for allowance: The examiner has further searched the prior art and has not discovered any prior art which fully teaches claims 8 and 13.
Regarding claim 8, the prior art of combination of the prior art fails to teach estimate that a task is being performed at the fastening location when a distance between a prescribed object and the virtual object is less than a second threshold, the second threshold being preset; and stop the estimation of the fastening location at which the task is being performed when a movement amount of the marker with respect to a reference position of the marker is greater than a first threshold, the reference position of the marker being acquired when the origin is set, the first threshold being preset. The closest prior art discovered is Grundhoefer et al. (US 20210097714 A1), Marchand (US 20210158587 A1), Sato et al. (US 20220230393 A), and Lee et al. (US 20110149042 A1). However, none of prior art references cited above, nor any other prior art discovered by the Examiner, fully teach claim 8, either singly or in an obvious combination. Therefore, claim 8 is distinguished over the prior art.
Claims 9 and 10 are also allowable over the prior art at least due to their respective dependencies.
Regarding claim 13, the prior art of combination of the prior art fails to teach
calculate a distance between a prescribed object and the virtual object; and when the distance is less than a second threshold, estimate that the task is being performed at the fastening location, the second threshold being preset. The closest prior art discovered is Grundhoefer et al. (US 20210097714 A1), Marchand (US 20210158587 A1), Sato et al. (US 20220230393 A), and Lee et al. (US 20110149042 A1). However, none of prior art references cited above, nor any other prior art discovered by the Examiner, fully teach claim 13, either singly or in an obvious combination. Therefore, claim 13 is distinguished over the prior art.
Claims 14-17 are also allowable over the prior art at least due to their respective dependencies.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hyorim Park/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights