CTNF 18/910,707 CTNF 87745 DETAILED ACTION The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Specification 06-11 AIA The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Claim Objections 07-29-01 AIA Claim 4 is objected to because of the following informalities: Claim 4 does not have clear antecedent basis for “the virtual objects.” Appropriate correction is required. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 1-3 and 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (US 2023/0104917) in view of Fukushima (US 2010/0309361) . Regarding claim 1 , Yamaguchi teaches/suggests: A mixed reality device capable of superimposing a virtual space on a real space (Yamaguchi [0043] “The wearable apparatus 7 is a device that a worker wears”), configured to: set a three-dimensional coordinate system in the virtual space based on a prescribed object imaged in the real space (Yamaguchi [0062] “obtaining dimension information such as the main dimensions for the tool RTn, the socket RTSn, etc., obtaining position information (for example, coordinates indicating a three-dimensional position) in real space for the tool RTn, the socket RTSn, etc., and transcribing the real space position information and dimension information to the virtual space, based on the virtual-space origin IO which corresponds one-to-one to the real-space origin RO”); display a virtual object at a predetermined position in the three-dimensional coordinate system (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects”); Yamaguchi does not teach/suggest: change a display direction of the virtual object according to a positional relationship between the virtual object and the mixed reality device. Fukushima, in view of Yamaguchi, teaches/suggests: change a display direction of the virtual object according to a positional relationship between the virtual object and the mixed reality device (Yamaguchi [0043] “The wearable apparatus 7 is a device that a worker wears” Fukushima [0059] “a planar object 710 is generated in a virtual 3D space and the position in the 3D space is changed to a position 710A in accordance with pieces of roll, pitch, and yaw information”). Fukushima further discloses in [0007]: “displaying a captured image to facilitate intuitive grasp of the direction in which the displayed image was captured.” Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the virtual objects of Yamaguchi to be displayed as taught/suggested by Fukushima to facilitate intuitive grasp of the direction. Regarding claim 2 , Yamaguchi as modified by Fukushima teaches/suggests: The mixed reality device according to claim 1, wherein the mixed reality device is configured to calculate a first vector along the display direction of the virtual object (Fukushima [0034] “The image captured direction can be expressed in more detail by continuously changing the ratio of the lengths of opposing sides in accordance with the angle defined by the reference direction and image captured direction” [The reference direction meets the first vector.] [0028] “The orientation (direction) is given by three types of values Roll, Pitch, and Yaw. Pitch is a rotation angle about the vector of gravity. Roll is a rotation angle about the vector of the baseline (for example, vector directed to the true north) on the horizontal plane. Yaw is a rotation angle about a vector (for example, vector directed to the due east) perpendicular to the vector of the baseline on the horizontal plane”), calculate a second vector connecting a position of the virtual object and a position of the mixed reality device (Yamaguchi [0043] “The wearable apparatus 7 is a device that a worker wears” Fukushima [0034] “The image captured direction can be expressed in more detail by continuously changing the ratio of the lengths of opposing sides in accordance with the angle defined by the reference direction and image captured direction” [In view of Yamaguchi and Fukushima, the direction of the wearable device meets the second vector.]), and change the display direction of the virtual object according to an angle between the first vector and the second vector (Fukushima [0059] “a planar object 710 is generated in a virtual 3D space and the position in the 3D space is changed to a position 710A in accordance with pieces of roll, pitch, and yaw information”). The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein. Regarding claim 3 , Yamaguchi as modified by Fukushima teaches/suggests: The mixed reality device according to claim 2, wherein the mixed reality device is configured to calculate a first angle between the first vector and the second vector in a first plane, the first plane being parallel to a first axis direction and a second axis direction of the three-dimensional coordinate system (Fukushima [0034] “The image captured direction can be expressed in more detail by continuously changing the ratio of the lengths of opposing sides in accordance with the angle defined by the reference direction and image captured direction” [0028] “The orientation (direction) is given by three types of values Roll, Pitch, and Yaw. Pitch is a rotation angle about the vector of gravity. Roll is a rotation angle about the vector of the baseline (for example, vector directed to the true north) on the horizontal plane. Yaw is a rotation angle about a vector (for example, vector directed to the due east) perpendicular to the vector of the baseline on the horizontal plane”), calculate a second angle between the first vector and the second vector in a second plane, the second plane being perpendicular to the first plane and parallel to the first vector (Fukushima [0034] “The image captured direction can be expressed in more detail by continuously changing the ratio of the lengths of opposing sides in accordance with the angle defined by the reference direction and image captured direction” [0028] “The orientation (direction) is given by three types of values Roll, Pitch, and Yaw. Pitch is a rotation angle about the vector of gravity. Roll is a rotation angle about the vector of the baseline (for example, vector directed to the true north) on the horizontal plane. Yaw is a rotation angle about a vector (for example, vector directed to the due east) perpendicular to the vector of the baseline on the horizontal plane”), and rotate the display direction of the virtual object by the first angle in the first plane and by the second angle in the second plane to change the display direction of the virtual object (Fukushima [0059] “a planar object 710 is generated in a virtual 3D space and the position in the 3D space is changed to a position 710A in accordance with pieces of roll, pitch, and yaw information”). The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein. Claim 14 recites limitation(s) similar in scope to those of claim 1, and is rejected for the same reason(s). Regarding claim 15 , Yamaguchi as modified by Fukushima teaches/suggests: A non-transitory computer-readable storage medium storing a program for causing a mixed reality device to execute the display control method according to claim 14 (Yamaguchi [0045] “The augmented reality server 12 provided in the wearable apparatus 7 is a computer that has a memory region for storing application software”). See also the treatment of claim 14 . 07-22-aia AIA Claim (s) 4-7 and 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (US 2023/0104917) in view of Fukushima (US 2010/0309361) as applied to claim 1 above, and further in view of Aoyama et al. (US 2015/0170422) . Regarding claim 4 , Yamaguchi as modified by Fukushima teaches/suggests: The mixed reality device according to claim 1, wherein the mixed reality device is configured to display a plurality of the virtual objects at a plurality of the predetermined positions in the three-dimensional coordinate system, respectively (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects”), Yamaguchi as modified by Fukushima does not teach/suggest: in a case where two or more of the plurality of virtual objects overlap, change a display position of at least one of the two or more virtual objects to reduce an overlap amount of the two or more virtual objects. Aoyama, however, teaches/suggests: in a case where two or more of the plurality of virtual objects overlap, change a display position of at least one of the two or more virtual objects to reduce an overlap amount of the two or more virtual objects (Aoyama [0032] “For drawing the drawing objects on the virtual screen on the basis of the set drawing positions, it is then determined whether an overlap of multiple drawing objects with different drawing priorities overlap occurs … the drawing position of a drawing object with a lower drawing priority may be moved”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify one or more of the virtual objects of Yamaguchi as modified by Fukushima to be moved as taught/suggested by Aoyama to reduce overlaps. Regarding claim 5 , Yamaguchi as modified by Fukushima and Aoyama teaches/suggests: The mixed reality device according to claim 4, wherein the mixed reality device is configured to move one of the two or more virtual objects with respect to the other of the two or more virtual objects to reduce the overlap amount (Aoyama [0032] “the drawing position of a drawing object with a lower drawing priority may be moved”). The same rationale to combine as set forth in the rejection of claim 4 above is incorporated herein. Regarding claim 6 , Yamaguchi as modified by Fukushima and Aoyama teaches/suggests: The mixed reality device according to claim 4, wherein the mixed reality device is configured to determine a priority of each of the two or more virtual objects (Aoyama [0032] “For drawing the drawing objects on the virtual screen on the basis of the set drawing positions, it is then determined whether an overlap of multiple drawing objects with different drawing priorities overlap occurs”), and move the virtual object with the lowest priority to reduce the overlap amount (Aoyama [0032] “the drawing position of a drawing object with a lower drawing priority may be moved”). The same rationale to combine as set forth in the rejection of claim 4 above is incorporated herein. Regarding claim 7 , Yamaguchi as modified by Fukushima and Aoyama teaches/suggests: The mixed reality device according to claim 4, wherein the mixed reality device is configured to set a virtual surface in front of the plurality of virtual objects (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects” Fukushima [0056] “the CPU 101 sets a virtual 3D space, and generates, in the space, a virtual planar object”), and project the plurality of virtual objects onto the virtual surface to calculate the overlap amount (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects” Fukushima [0056] “the CPU 101 maps the image captured by the image capturing unit 113 on the planar object” Aoyama [0032] “For drawing the drawing objects on the virtual screen on the basis of the set drawing positions, it is then determined whether an overlap of multiple drawing objects with different drawing priorities overlap occurs”). The same rationales to combine as set forth in the rejection of claims 1 and 4 above are incorporated herein. Regarding claim 9 , Yamaguchi as modified by Fukushima and Aoyama teaches/suggests: The mixed reality device according to claim 4, wherein the plurality of virtual objects are respectively displayed corresponding to a plurality of fastening locations in the real space where screws are turned (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects”). Regarding claim 10 , Yamaguchi as modified by Fukushima and Aoyama teaches/suggests: The mixed reality device according to claim 9, wherein on each of the plurality of virtual objects, at least one selected from the group consisting of identification information of the corresponding fastening location, a torque value required for the corresponding fastening location, a torque value detected by a tool turning the screw, and a number of times the screw has been tightened for the corresponding fastening location is displayed (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects” [0099] “only the fastening torque lower-limit value Tr_Min (Nm), fastening target torque value Tr_Set (Nm), and the fastening torque upper-limit value Tr_Max (Nm), which are for the bolts RBn (n = 1 to 4) and are obtained from the work instruction document…”) . 07-22-aia AIA Claim (s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (US 2023/0104917) in view of Fukushima (US 2010/0309361) and Aoyama et al. (US 2015/0170422) as applied to claim 4 above, and further in view of Noda et al. (US 2023/0047470) . Regarding claim 8 , Yamaguchi as modified by Fukushima and Aoyama does not teach/suggest: The mixed reality device according to claim 4, wherein the mixed reality device is configured to differentiate a display of the virtual object that overlaps with a wearer’s line of sight from a display of the other virtual objects among the plurality of virtual objects . Adachi, however, teaches/suggests differentiate a display of the virtual object that overlaps with a wearer’s line of sight from a display of the other virtual objects among the plurality of virtual objects (Noda [0147] “when there is, for example, the virtual object 5 selected on the basis of the correction line-of-sight vector, the selected virtual object 5 is, for example, highlighted to be displayed to indicate that the object has been selected ”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify one or more of the virtual objects of Yamaguchi as modified by Fukushima and Aoyama to be highlighted as taught/suggested by Noda to indicate selection . 07-21-aia AIA Claim (s) 11-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (US 2023/0104917) in view of Aoyama et al. (US 2015/0170422) . Regarding claim 11 , Yamaguchi teaches/suggests: A mixed reality device capable of superimposing a virtual space on a real space (Yamaguchi [0043] “The wearable apparatus 7 is a device that a worker wears”), configured to: set a three-dimensional coordinate system in the virtual space based on a prescribed physical object imaged in the real space (Yamaguchi [0062] “obtaining dimension information such as the main dimensions for the tool RTn, the socket RTSn, etc., obtaining position information (for example, coordinates indicating a three-dimensional position) in real space for the tool RTn, the socket RTSn, etc., and transcribing the real space position information and dimension information to the virtual space, based on the virtual-space origin IO which corresponds one-to-one to the real-space origin RO”); display a plurality of virtual objects at a plurality of positions in the three-dimensional coordinate system, respectively (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects”); Yamaguchi does not teach/suggest: in a case where two or more of the plurality of virtual objects overlap, change a display position of at least one of the two or more virtual objects to reduce an overlap amount between the two or more virtual objects. Aoyama, however, teaches/suggests: in a case where two or more of the plurality of virtual objects overlap, change a display position of at least one of the two or more virtual objects to reduce an overlap amount between the two or more virtual objects (Aoyama [0032] “For drawing the drawing objects on the virtual screen on the basis of the set drawing positions, it is then determined whether an overlap of multiple drawing objects with different drawing priorities overlap occurs … the drawing position of a drawing object with a lower drawing priority may be moved”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify one or more of the virtual objects of Yamaguchi to be moved as taught/suggested by Aoyama to reduce overlaps. Regarding claim 12 , Yamaguchi as modified by Aoyama teaches/suggests: The mixed reality device according to claim 11, wherein the plurality of virtual objects are respectively displayed corresponding to a plurality of fastening locations in the real space where screws are turned (Yamaguchi [0146] “the server captures and saves an image resulting from writing a work result (worker ID, work date, work time, fastening complete torque value Tr_rslt, work instruction document, etc.) near the bolts RBn (n = 1 to 4) in the augmented reality space which includes fastening work target objects”). Regarding claim 13 , Yamaguchi as modified by Aoyama teaches/suggests: The mixed reality device according to claim 11, wherein the mixed reality device is configured to move one of the two or more virtual objects with respect to the other of the two or more virtual objects to reduce the overlap amount (Aoyama [0032] “the drawing position of a drawing object with a lower drawing priority may be moved”). The same rationale to combine as set forth in the rejection of claim 11 above is incorporated herein . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure : US 11688151 – dynamic labels in ER US 2013/0031511 – move virtual objects overlapping FOV US 2025/0076971 – interactive viewing of 3D content Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANH-TUAN V NGUYEN whose telephone number is 571-270-7513. The examiner can normally be reached on M-F 9AM-5PM ET. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached on 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ANH-TUAN V NGUYEN/ Primary Examiner, Art Unit 2619 Application/Control Number: 18/910,707 Page 2 Art Unit: 2619 Application/Control Number: 18/910,707 Page 3 Art Unit: 2619 Application/Control Number: 18/910,707 Page 4 Art Unit: 2619 Application/Control Number: 18/910,707 Page 5 Art Unit: 2619 Application/Control Number: 18/910,707 Page 6 Art Unit: 2619 Application/Control Number: 18/910,707 Page 7 Art Unit: 2619 Application/Control Number: 18/910,707 Page 8 Art Unit: 2619 Application/Control Number: 18/910,707 Page 9 Art Unit: 2619 Application/Control Number: 18/910,707 Page 10 Art Unit: 2619 Application/Control Number: 18/910,707 Page 11 Art Unit: 2619 Application/Control Number: 18/910,707 Page 12 Art Unit: 2619 Application/Control Number: 18/910,707 Page 13 Art Unit: 2619 Application/Control Number: 18/910,707 Page 14 Art Unit: 2619 Application/Control Number: 18/910,707 Page 15 Art Unit: 2619
Prosecution Insights