CTNF 18/882,426 CTNF 101726 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Information Disclosure Statement The information disclosure statement (IDS) submitted on 02/07/2025, 10/9/2024, and 9/11/24 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15-03-aia AIA Claim s 1-4 and 11-12 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Grundhoefer et al. (US 20210097714 A1) (Hereinafter Grundhoefer) . Regarding claim 1, Grundhoefer discloses a mixed reality device, configured to: (para. [0034], “Examples of CGR include virtual reality and mixed reality “; para. [0040], “There are many different types of electronic systems that enable a person to sense and/or interact with various CGR environments. Examples include head mounted systems, projection-based systems, heads-up displays (HUDs),”) set an origin of a virtual space by using a marker located in a real space; (para. [0053], “In some implementations, the location module 340 includes a visual marker detection unit 342, a relative positioning unit 344, and a location determination unit 346. The visual marker detection unit 342 is configured to detect and identify visual markers in images of a physical environment. The relative positioning unit 344 is configured to determine relative positioning to a detected object or visual marker from images of the physical environment. The location determination unit 346 is configured to determine a calculated location of objects given a geolocation (crowd-sourced or signal-based positioning) and determined relative positionings.”) display a virtual object at a preset position in a three-dimensional coordinate system, the three-dimensional coordinate system being based on the origin ; (para. [0032], “In CGR, a subset of a person's physical motions, or representations thereof, are tracked, and, in response, one or more characteristics of one or more virtual objects simulated in the CGR environment are adjusted in a manner that comports with at least one law of physics. For example, a CGR system may detect a person's head turning and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment.”) acquire a movement amount of the marker with respect to a reference position of the marker, the reference position of the marker being acquired when the origin is set; and (para. [0092], “In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.). move the origin according to the movement amount when the movement amount is greater than a first threshold, the first threshold being preset. (para. [0092], “In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted , etc.).”) Regarding claim 2 , Grundhoefer discloses the device according to claim 1, further configured to: recognize the marker based on an image of the marker, the origin being set by using a position and an orientation of the marker as a reference. (para. [0061], “In some implementations, the relative positioning determines the pose (e.g., position and orientation ) of the detected visual marker 410 relative to the pose of the electronic device 420.”) Regarding claim 3, Grundhoefer discloses the device according to claim 2, further configured to: reset the origin when the marker is reimaged after the moving of the origin, the resetting being performed by using a position and an orientation of the marker recognized based on an image captured by the reimaging as a reference. (para. [0061], As shown in FIG. 4 and FIG. 5, the electronic device 420 can determine a relative positioning between the electronic device 420 and the visual marker 410 based on the image or images (e.g., the CGR environments 405A) of the physical environment 405 including the visual marker 410. In some implementations, the relative positioning determines the relative orientation of the visual marker 410 with respect to the electronic device 420.”) Regarding claim 4 , Grundhoefer discloses the device according to claim 1, further configured to: repeatedly acquire the movement amount; and determine whether or not the movement amount is greater than the first threshold each time the movement amount is acquired. (para. [0092], “In some implementations, metadata associated with the visual marker identifies the visual marker as a moving visual marker and the associated geolocation of the moving visual marker is updated based on a time threshold or a position change threshold (e.g., with older observed locations being discarded or de-weighted, large differences in observed locations suggesting the visual marker has been moved and previous observations should be discarded or de-weighted, etc.). In some implementations, each changeable geolocation of the moving visual marker is crowd-sourced.”) Regarding claim 11, similar reasoning as discussed in claim 1 is applied. Regarding claim 12, claim 12 is a non- transitory computer readable storage medium configured to store a program (para. [0012], “In accordance with some implementations, a non-transitory computer readable storage medium has stored therein instructions, which, when executed by one or more processors of a device, cause the device to perform or cause performance of any of the methods described herein.”) and similar reasoning as discussed in claim 11 is applied . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Grundhoefer et al. (US 20210097714 A1) (Hereinafter Grundhoefer) in view of Marchand (US 20210158587 A1) (Hereinafter Marchand) . Regarding claim 5, Grundhoefer does not explicitly disclose the device according to claim 1, further configured to: calculate the movement amount by using an image of an article on which a task is performed, or by using a detection result from a sensor fixed with respect to the article. However, Marchand more explicitly teaches, in the context of mixed reality device, The device according to claim 1, further configured to: calculate the movement amount by using an image of an article on which a task is performed, or by using a detection result from a sensor fixed with respect to the article. (Marchand, para [0014], “In association with either or both of the first and second spatial-positioning subsystems, algorithmic triangulation may be employed with the aid of sensors , transducers, transmitters, and receivers that transmit and/or receive electromagnetic energy including, by way of example, radio frequencies and/or light, such as infrared.”; Marchand, para. [0017], “Virtual-reality and mixed-reality headsets employ various on-board sensors such as, but way of example, an inertial measurement unit (IMU), which may include an accelerometer, gyroscope, and a magnetometer; “environment understanding” sensors; one or more cameras; and one or more light-sensors”; Marchand, para. [0020], “Alternatively or additionally, tool workpiece object identification can be facilitated with camera imagery acquired by one or more cameras carried by the headset or otherwise located to acquire images of objects within the physical space.”) As both Grundhoefer and Marchand are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the device according to claim 1, further configured to: calculate the movement amount by using an image of an article on which a task is performed, or by using a detection result from a sensor fixed with respect to the article , in the context of mixed reality device, by Grundhoefer according to the teaching of Marchand in order to increase the efficiency of the user and to reduce procedural errors (Marchand, para. [0006]). Regarding claim 6, Grundhoefer does not explicitly disclose the device according to claim 1, wherein the virtual object is displayed to correspond to a fastening location of an article on which a task is performed. However, Marchand more explicitly teaches, in the context of mixed reality device, the device according to claim 1, wherein the virtual object is displayed to correspond to a fastening location of an article on which a task is performed (Marchand, 350 in FIG 4B; para. [0054], “illustrative machine-generated indicia 350 are presented to the system user HSO wearing the headset 300 in accordance with a current procedure step 135 as dictated by the machine-executable procedure program 130.). PNG media_image1.png 302 434 media_image1.png Greyscale As both Grundhoefer and Marchand are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the device according to claim 1, wherein the virtual object is displayed to correspond to a fastening location of an article on which a task is performed , in the context of mixed reality device, by Grundhoefer according to the teaching of Marchand in order to increase the efficiency of the user and to reduce procedural errors (Marchand, para. [0006]) . 07-21-aia AIA Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Grundhoefer et al. (US 20210097714 A1) (Hereinafter Grundhoefer) in view of Marchand (US 20210158587 A1) (Hereinafter Marchand) and further in view of Yoshii et al. (US 20230326096 A1) (Hereinafter Yoshii) . Regarding claim 7, Grundhoefer in view of Marchand further teaches when the distance is less than a second threshold, estimate that the task is being performed at the fastening location, the second threshold being preset. (Marchand, para. [0058], “there is established a predefined proximity P pd between a work piece object 200 and the smart tool 250 relative to which the smart tool 250 is selectively activatable and deactivatable.”; Marchand, para. [0059], “According to one version, in response to a determination that the real-time proximity P RT is within the predefined proximity P PD , the data-processing system 110 is caused to communicate to the smart tool 250 a command to activate.”) However, Grundhoefer in view of Marchand does not explicitly disclose the device according to claim 6, further configured to: calculate a distance between a prescribed object and the virtual object. Yoshii more explicitly teaches the device according to claim 6, further configured to: calculate a distance between a prescribed object and the virtual object (Yoshii, para. [0087], “The processing device 10 repeats the determination of whether or not the prescribed physical object contacts the first object (step S4).”; Yoshii, para. [0019], “The first object is virtual”). As Grundhoefer, Marchand, and Yoshii are from the same field of endeavor, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to include the device according to claim 6, further configured to: calculate a distance between a prescribed object and the virtual object , in the context of mixed reality device, by Grundhoefer in view of Marchand according to the teaching of Yoshii in order to maintain acceptable distance and reduce procedural errors . 07-21-aia AIA Claims 8-10 a re rejec ted under 35 U.S.C. 103 as being unpatentable over March and (US 20210158587 A1) (Hereinafter Marchand) in view of Yoshii et al. (US 20230326096 A1) (Hereinafter Yoshii). Regar ding claim 8 , Marchand discloses A mixed reality device, configured to: (Marchand, ABSTRACT, “a mixed-reality headset”) set an origin of a virtual space by using a marker located in a real space; (FIG. 3; para. [0046], “The calibration cluster CC includes three markers or reference anchors 460, 560, and A R. First and second reference anchors 460 and 560 are associated with, respectively, the first and second spatial-positioning systems 400 and 500 indicated in FIGS. 1 and 2. A third reference anchor A R is provided for orienting the working end (e.g., a bit or socket—“tool point”) of a power tool 250, for example, and may also be associated with the second spatial-positioning system 500.") PNG media_image2.png 663 550 media_image2.png Greyscale display a virtual object to correspond to a fastening location of an article; (Marchand, 350 in FIG. 4B; para. [0054], “illustrative machine-generated indicia 350 are presented to the system user HSO wearing the headset 300 in accordance with a current procedure step 135 as dictated by the machine-executable procedure program 130.”) PNG media_image1.png 302 434 media_image1.png Greyscale estimate that a task is being performed at the fastening location when a distance between a prescribed object and the virtual object is less than a second threshold, the second threshold being preset; and (Marchand, para. [0058], “there is established a predefined proximity P pd between a work piece object 200 and the smart tool 250 relative to which the smart tool 250 is selectively activatable and deactivatable.”; Marchand, para. [0059], “According to one version, in response to a determination that the real-time proximity P RT is within the predefined proximity P PD , the data-processing system 110 is caused to communicate to the smart tool 250 a command to activate .”) stop the estimation of the fastening location at which the task is being performed when a movement amount of the marker with respect to a reference position of the marker is greater than a first threshold, the reference position of the marker being acquired when the origin is set, the first threshold being preset. (Marchand, para. [0059], “Correlatively, in another version, in response to a determination that the real-time P RT proximity is outside of the predefined proximity P PD , the data-processing system 110 is caused to communicate to the smart tool a command to deactivate .”; Marchand, para. [0051], “the reference characters specifically referencing object tags 522 associated with workpiece objects 200 may further include the alphabetic characters WP, such that the full alphanumeric reference character pertaining thereto is 522 wp . Correlatively, the reference characters designating object tags 522 associated with tools 250 may further include the alphabetic designator T, such that the full alphanumeric reference character pertaining thereto is 522 T .”) Marchand does not explicitly disclose the virtual object. However, Yoshii more explicitly teaches, in the context of mixed reality device, the virtual object (Yoshii, para. [0087], “The processing device 10 repeats the determination of whether or not the prescribed physical object contacts the first object (step S4).”; Yoshii, para. [0019], “The first object is virtual”). As both Marchand and Yoshii are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to take the virtual object to calculate the distance, in the context of mixed reality device, by Marchand according to the teaching of Yoshii in order to increase the efficiency of the user and to reduce procedural errors. Regarding claim 9 , Marchand in view of Yoshii further teaches the device according to claim 8, further configured to: restart the estimation of the fastening location at which the task is being performed when the origin is reset after the stop of the estimation. (Marchand, para. [0058], “From this location data, a proximity comparator 180 associated with the data-processing system 110 can algorithmically determine— on a continuous basis or in predetermined temporal increments —the relative real-time proximity P RT of the workpiece object 200 and the smart tool 250. With the real-time proximity P RT of the workpiece object 200 and the smart tool 250 determined, the real-time proximity PRT is algorithmically compared to the predefined proximity P PD .”) Regarding claim 10 , Marchand in view of Yoshii further teaches the device according to claim 8, further configured to: repeatedly acquire the movement amount; and determine whether or not the movement amount is greater than the first threshold each time the movement amount is acquired. (Marchand, para. [0058], “From this location data, a proximity comparator 180 associated with the data-processing system 110 can algorithmically determine— on a continuous basis or in predetermined temporal increments —the relative real-time proximity P RT of the workpiece object 200 and the smart tool 250. With the real-time proximity P RT of the workpiece object 200 and the smart tool 250 determined, the real-time proximity PRT is algorithmically compared to the predefined proximity P PD .”) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Hyorim Park/Examiner, Art Unit 2615 /ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615 Application/Control Number: 18/882,426 Page 2 Art Unit: 2619 Application/Control Number: 18/882,426 Page 3 Art Unit: 2619 Application/Control Number: 18/882,426 Page 4 Art Unit: 2619 Application/Control Number: 18/882,426 Page 5 Art Unit: 2619 Application/Control Number: 18/882,426 Page 6 Art Unit: 2619 Application/Control Number: 18/882,426 Page 7 Art Unit: 2619 Application/Control Number: 18/882,426 Page 8 Art Unit: 2619 Application/Control Number: 18/882,426 Page 9 Art Unit: 2619 Application/Control Number: 18/882,426 Page 10 Art Unit: 2619 Application/Control Number: 18/882,426 Page 11 Art Unit: 2619 Application/Control Number: 18/882,426 Page 12 Art Unit: 2619 Application/Control Number: 18/882,426 Page 13 Art Unit: 2619 Application/Control Number: 18/882,426 Page 14 Art Unit: 2619
Prosecution Insights