CTNF 18/973,716 CTNF 101769 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Priority Receipt is acknowledged that application claims priority to foreign application with application number JP2022-094576 dated 06/10/2022. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78. Information Disclosure Statement The IDS dated December 09, 2024 has been considered and placed in the application file. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre- AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: a construction unit that constructs , in a virtual space, a three-dimensional (3D) model of a target object to be worked on by the worker in a real space in claim 16; a first acquisition unit that acquires position information and orientation information of the worker in the real space in claim 16; a synchronization unit that synchronizes a position and an orientation of a virtual camera in the virtual space with a position and an orientation of the worker based on the position information and the orientation information in claim 16; a generation unit that generates a virtual camera video indicating the 3D model captured by the virtual camera synchronized in claim 16; and an output unit that outputs output information for displaying an output image including the virtual camera video in claim 16; Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. Claims 1-11 and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable and obvious over US Patent Application Publication US 2017/0076491 A1, (JIANG et al.) (hereinafter “Jiang”) in view of US Patent Application Publication US 2021/0272329 A1, (Katsumata et al.) (hereinafter “Katsumata”). Regarding claim 1, Jiang teaches an information processing method in an information processing system that supports a work of a worker, the method comprising: (Jiang Abstract, “[0009] … operation Support method , including generating a three dimensional panorama image by overlapping multiple images with each other based on posture information of a camera and a feature point map of the multiple images captured by the camera, and displaying the three dimensional panorama image at a first display device; and outputting, at a second display device, position information of a target indicated based on current posture information of the camera in response to an indication of the target on the three dimensional panorama image.”) acquiring position information and orientation information of the worker in the real space; (Jiang “[0032] An image frame 2c , which includes a two dimensional (2D) image captured by the camera 21 of the operator 2, is sent as a site state to the remote Support apparatus 101…”; “[0044-0045] By displaying the 3D panorama image 4 as the scene of the view direction of the operator 2, it is possible for the instructor 1 to intuitively recognize a head position and the view direction of the operator 2. That is, since the instructor 1 may physically sense the scene, which is viewed from the head position of the operator 2, it is possible for the instructor 1 to acquire a realistic feeling of a distance between a target 7t and the operator 2 more precisely…precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. ”; “[0057] The IMU 215 includes an inertial sensor , and also, corresponds to a device that includes an acceleration measuring function and a gyro function. The IMU 215 outputs information pertinent to movement of the operator 2 …”; “[0058] The short distance radio communicating part 216 conducts short distance radio communications with each of the display device 21d and the camera 21c . The short distance communication may be Bluetooth (registered trade mark) or the like. The network communication part 217 sends the image frame 2c, to which posture information is attached , to the remote Support apparatus 101…”) synchronizing a position and an orientation of a virtual camera in the virtual space with a position and an orientation of the worker based on the position information and the orientation information; (Jiang “[0061] In the operator terminal 201, a monocular Simultaneous Localization And Mapping (SLAM) function…By the monocular SLAM function, information of the position and the posture of the camera 21 (hereinafter, called “position-and-posture information 3c (FIG. 3)) …are simultaneously estimated from the image captured by the camera 21 …”; “[0062] By acquiring the position-and-posture information 3c …it is possible to generate an environment map representing a real environment in the three dimensions. A self-position estimation is conducted by the monocular SLAM function based on world coordinate systems .”; “[0069] …The camera parameter K is generally estimated by using a calibration chart . Alternatively, the skew s=0 and the aspect ratio a-1 may be defined, and the c, and c, may indicate an image center.”) generating a virtual camera video … captured by the virtual camera synchronized; and (Jiang “[0060] The camera 21c includes the short distance wireless communication function. The camera 21c is mounted on the head of the operator 2, captures a video in the visual line direction of the operator 2, and sends the image frames 2c to the operator terminal 201 by the short distance wireless communication.”) outputting output information for displaying an output image including the virtual camera video (Jiang “[0059] … display device 21d may be a wearable-type display device being eyeglasses mounted towards the visual line direction on the head. The display device 21d includes a transparent display part. It is preferable for the operator 2 to visually observe a real view in the visual line direction. The display device 21d displays the instruction detail 2g included in the instruction information 2f received from the operator terminal 201 by the short distance wireless communication.”; “[0079] …the operator 1 operates the input device 114 and points to the target 7t on the 3D panorama image 4 displayed at the display device 115 of the remote Support apparatus 101. The instruction detail 2g is displayed at a precise position with respect to the target 7t at the display device 21d. A method for acquiring a display position of the instruction detail 2g will be described for a case in which the target 7t is inside the visual field 7d and a case in which the target 7t is outside the visual field 7d…”; “[0060] The camera 21c includes the short distance wireless communication function. The camera 21c is mounted on the head of the operator 2, captures a video in the visual line direction of the operator 2, and sends the image frames 2c to the operator terminal 201 by the short distance wireless communication.”) However, Jiang is silent about constructing, in a virtual space, a three-dimensional (3D) model of a target object to be worked on by the worker in a real space; indicating the 3D model. Katsumata teaches constructing, in a virtual space, a three-dimensional (3D) model of a target object to be worked on by the worker in a real space; indicating the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang and Katsumata are analogous art as both of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by constructing, in a virtual space, a three-dimensional (3D) model of a target object to be worked on by the worker in a real space; indicating the 3D model as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Claim 16 is directed to an information processing device that supports a work of a worker, the information processing device comprising: (Jiang “[0037] The operator terminal 201 is regarded as an information processing terminal such as a smart device or the like, and includes various communication functions. The display device 21d may be a device such as the HMD , which is wearable and capable of input and output voice.”; “[0046] FIG. 2 is a diagram illustrating a hardware configuration of the remote operation Support system. In the remote operation support system 1000, the remote support apparatus 101 includes a Central Processing Unit (CPU) 111 , a memory 112 , a Hard Disk Drive (HDD) 113, an input device 114, a display device 115, an audio input/output device 116, a network communication part 117, and a drive device 118. At least one of the memory 112 and the Hard Disk Drive (HDD) 113 corresponds to a storage device 130 .”) and its scope and functions are substantially similar to the steps performed by the method claim 1 and therefore claim 16 is also rejected with the same rationale as specified in the rejection of claim 1. Claim 17 is directed to a non-transitory computer readable recording medium storing an information processing program that causes a computer to function as an information processing system that supports a work of a worker, the program causing the computer to execute processing of: (Jiang “[0037] The operator terminal 201 is regarded as an information processing terminal such as a smart device or the like, and includes various communication functions. The display device 21d may be a device such as the HMD , which is wearable and capable of input and output voice.”; “[0046] FIG. 2 is a diagram illustrating a hardware configuration of the remote operation Support system. In the remote operation support system 1000, the remote support apparatus 101 includes a Central Processing Unit (CPU) 111 , a memory 112 , a Hard Disk Drive (HDD) 113, an input device 114, a display device 115, an audio input/output device 116, a network communication part 117, and a drive device 118. At least one of the memory 112 and the Hard Disk Drive (HDD) 113 corresponds to a storage device 130 .”; “[0054] The recording medium 119 may be formed of a non-transitory or tangible computer-readable recording medium including a structure…”) and its scope and functions are substantially similar to the steps performed by the method claim 1 and therefore claim 17 is also rejected with the same rationale as specified in the rejection of claim 1. Regarding claim 2, Jiang teaches wherein the position information and the orientation information are real marker information indicating a position and an orientation of a real marker worn by the worker, and (Jiang “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”) the method further comprises: acquiring initial marker information indicating a position and an orientation of the real marker installed in the real space; (Jiang “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”) executing spatial calibration that associates the real space with the virtual space based on the initial marker information and a relative position of the target object and the real marker; (Jiang Figs. 10A-10C and Fig. 11, “[0095] Since the angle difference 0d is sent to the operator terminal 201, it is possible for the operator terminal 201 to relatively easily s pecify the target 7t based on coordinates of the feature point 3p and a direction vector. After the target 7t is specified, in association with the position and the posture of the head of the operator 2 …”; “[0098] When the target 7t is indicated by the instructor 1 in the visual field 7d-5, as described above, the operator terminal 201 specifies the anchor feature point 3pa, and recognizes the target 7t by multiple feature points 3p including the anchor feature point 3pa .”; “[0099] The operator terminal 201 conducts a coordinate conversion from the map coordinate system 3s to the screen coordinate system 21s, and calculates a display position 2h-5 to display the instruction detail 2g. In this example, each of distances from the target 7t to obstacles (edges of the visual field 7d-5) around the target 7t is compared with a size of the instruction detail 2g, and the display position 2h-5 is specified from excess areas around the target 7t…”; “[0125] The target posture acquisition part 283 acquires the angle difference 0d of the target 7t from information reported from the instruction information analysis part 271, and provides the acquired angle difference 0d to the display location determination part 287.”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”; “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”) acquiring the real marker information indicating the position and the orientation of the real marker worn by the worker in the real space; and (Jiang “[0125] Also, the head posture acquisition part 285 acquires head posture information of the operator 2 from information reported from the head posture estimation part 277, and provides the acquired head posture information to the display location determination part 287. The head posture information includes location coordinates of the operator 2 in the three dimensional real environment and posture information indicated by (0 pitch, 0 yaw).”; “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p- 6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”) synchronizing the position and the orientation of the virtual camera with the position and the orientation of the real marker indicated by the real marker information based on the real marker information (Jiang “[0060] The camera 21c includes the short distance wireless communication function. The camera 21c is mounted on the head of the operator 2, captures a video in the visual line direction of the operator 2, and sends the image frames 2c to the operator terminal 201 by the short distance wireless communication.”; “[0061] In the operator terminal 201, a monocular Simultaneous Localization And Mapping (SLAM) function…By the monocular SLAM function, information of the position and the posture of the camera 21 (hereinafter, called “position-and-posture information 3c (FIG. 3)) …are simultaneously estimated from the image captured by the camera 21 …”; “[0062] By acquiring the position-and-posture information 3c …it is possible to generate an environment map representing a real environment in the three dimensions. A self-position estimation is conducted by the monocular SLAM function based on world coordinate systems .”; “[0069] …The camera parameter K is generally estimated by using a calibration chart . Alternatively, the skew s=0 and the aspect ratio a-1 may be defined, and the c, and c, may indicate an image center.”; “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”) Regarding claim 3, Jiang teaches wherein the spatial calibration includes installing a virtual marker corresponding to the real marker at an arbitrary position in the virtual space based on the initial marker information, (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”) …with reference to the virtual marker so as to maintain a relative positional relationship between the real marker and the target object based on the relative position, and (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) generating calibration information including a real reference vector indicating a relative position of the real marker with respect to the target object when the spatial calibration is executed and a virtual reference vector indicating a relative position of the virtual marker (Jiang “[0082] First, a vector representing the view direction toward the 3D image 3dg-2 is set as a reference vector 7r-2 (FIG. 7C) based on a position 8p–2 of the camera 21c in the three dimensional space.”; “[0088] First, a vector representing the view direction toward the 3D image 3dg-2 is set as the reference vector 7r-2 (FIG. 8C) based on the position 8p–2 of the camera 21c in the three dimensional space.”; “[0094] The remote support apparatus 101 determines an angle as the angle difference 0d between a line connecting the anchor feature point 3pa with the position 8p and the vector 7c toward center coordinates of the image 2dg from the position 8p. The instruction position 7p matches with the anchor feature point 3pa. The angle difference 0d is represented by an angle difference between the vector 7v and the vector 7C .”) However, Jiang is silent about installing the 3D model in the virtual space…with respect to the 3D model. Katsumata teaches installing the 3D model in the virtual space…with respect to the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang and Katsumata are analogous art as both of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by installing the 3D model in the virtual space…with respect to the 3D model as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Regarding claim 4, Jiang teaches wherein the synchronizing includes calculating a real space vector indicating a relative position of the real marker with respect to the target object based on the real marker information, (Jiang “[0082] First, a vector representing the view direction toward the 3D image 3dg-2 is set as a reference vector 7r-2 (FIG. 7C ) based on a position 8p–2 of the camera 21c in the three dimensional space. ”; “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”) calculating a difference vector between the real space vector and the real reference vector, (Jiang “[0083] Accordingly, the angle difference 0d a target vector 7v-2 from the position 7p-2 of the camera 21c to the instruction position 7p-2 is calculated . The angle difference 0d indicates a rotation amount of the head of the operator 2.”; “[0082] First, a vector representing the view direction toward the 3D image 3dg-2 is set as a reference vector 7r-2 (FIG. 7C) based on a position 8p–2 of the camera 21c in the three dimensional space.”; “[0092] The remote support apparatus 101 calculates a target vector 7v by using a position 8p of the camera 21c (the head of the operator 2) acquired from the position-and posture information 3c. By the calculated target vector 7v, a view point 9p is acquired in a direction of the target vector 7v…”; “[0094] The remote support apparatus 101 determines an angle as the angle difference 0d between a line connecting the anchor feature point 3pa with the position 8p and the vector 7c toward center coordinates of the image 2dg from the position 8p. The instruction position 7p matches with the anchor feature point 3pa. The angle difference 0d is represented by an angle difference between the vector 7v and the vector 7c .”) determining a position of the virtual marker with respect to… (Jiang “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) …based on the virtual reference vector and the difference vector and determining an orientation of the virtual marker based on the orientation of the real marker included in the real marker information (Jiang “[0083] Accordingly, the angle difference 0d a target vector 7v-2 from the position 7p-2 of the camera 21c to the instruction position 7p-2 is calculated . The angle difference 0d indicates a rotation amount of the head of the operator 2.”; “[0082] First, a vector representing the view direction toward the 3D image 3dg-2 is set as a reference vector 7r-2 (FIG. 7C) based on a position 8p–2 of the camera 21c in the three dimensional space.”; “[0092] The remote support apparatus 101 calculates a target vector 7v by using a position 8p of the camera 21c (the head of the operator 2) acquired from the position-and posture information 3c. By the calculated target vector 7v, a view point 9p is acquired in a direction of the target vector 7v…”; “[0094] The remote support apparatus 101 determines an angle as the angle difference 0d between a line connecting the anchor feature point 3pa with the position 8p and the vector 7c toward center coordinates of the image 2dg from the position 8p. The instruction position 7p matches with the anchor feature point 3pa. The angle difference 0d is represented by an angle difference between the vector 7v and the vector 7c .”; “[0125] Also, the head posture acquisition part 285 acquires head posture information of the operator 2 from information reported from the head posture estimation part 277, and provides the acquired head posture information to the display location determination part 287. The head posture information includes location coordinates of the operator 2 in the three dimensional real environment and posture information indicated by (0 pitch, 0 yaw).”; “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”) determining the position of the virtual marker as the position of the virtual camera, and (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”) setting the orientation of the virtual marker as the orientation of the virtual camera (Jiang Fig. 1, “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f…”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) However, Jiang is silent about the 3D model. Katsumata teaches the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang and Katsumata are analogous art as both of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by the 3D model as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Regarding claim 5, Jiang teaches wherein the initial marker information and the real marker information are information acquired by using a method of sensing a surrounding environment and detecting a relative position of the initial marker information and the real marker information (Jiang “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”; “[0125] Also, the head posture acquisition part 285 acquires head posture information of the operator 2 from information reported from the head posture estimation part 277, and provides the acquired head posture information to the display location determination part 287. The head posture information includes location coordinates of the operator 2 in the three dimensional real environment and posture information indicated by (0 pitch, 0 yaw).”; “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”; “[0057] The IMU 215 includes an inertial sensor , and also, corresponds to a device that includes an acceleration measuring function and a gyro function. The IMU 215 outputs information pertinent to movement of the operator 2.”) Regarding claim 6, Jiang teaches wherein the spatial calibration includes installing a virtual marker corresponding to the real marker in the virtual space based on the initial marker information, (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”) However, Jiang is silent about installing the 3D model in the virtual space so as to maintain the relative positional relationship between the real marker and the target object based on the relative position, and generating, as the calibration information, a coordinate of the 3D model installed. Katsumata teaches installing the 3D model in the virtual space so as to maintain the relative positional relationship between the real marker and the target object based on the relative position, and (Katsumata Fig. 2, “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”; “[0026] …the index used for acquiring the position of the imaging display apparatus 110 is , for example , an index in a quadrangle shape ( hereinafter referred to as a quadrangle index P1 ) as illustrated in FIG . 2 , and a plurality of quadrangle indexes P1 is arranged in the three - dimensional real space…”) generating, as the calibration information, a coordinate of the 3D model installed (Katsumata Fig. 2, “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang and Katsumata are analogous art as both of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by installing the 3D model in the virtual space so as to maintain the relative positional relationship between the real marker and the target object based on the relative position, and generating, as the calibration information, a coordinate of the 3D model installed as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Regarding claim 7, Jiang teaches wherein the synchronizing includes determining the position of the real marker as the position of the virtual camera, and determining the orientation of the real marker as the orientation of the virtual camera (Jiang Fig. 1, “[0037-0039] The display device 21d may be a device such as the HMD , which is wearable and capable of input and output voice… The camera 21c may be a device such as a head mounted camera (HMC)…”; “[0061-0062] By the monocular SLAM function, information of the position and the posture of the camera 21 (hereinafter, called “position-and-posture information 3c”…”) Regarding claim 8, Jiang teaches wherein the initial marker information and the real marker information are information acquired by using a sensor representing a position of an object by using a coordinate system of the real space (Jiang “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”; “[0062] By acquiring the position-and-posture information 3c and the feature point map 3m, it is possible to generate an environment map representing a real environment in the three dimensions. A self-position estimation is conducted by the monocular SLAM function based on world coordinate systems. ”; “[0102] The map coordinate system 3s of the feature map 3m represents a coordinate system of a real environment (that is, a coordinate system in a three dimensional real space )…”; “[0045] …it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2…”) Regarding claim 9, Jiang teaches wherein the initial marker information includes first initial marker information indicating a position and an orientation of a first real marker installed at a position different from a position of the target object, (Jiang “[0103] In FIG. 11, the movement example of the operator 2 in FIG. 10A is referred to. The map coordinate system 3s of the feature point map 3 is defined as the world coordinate system. Locations 8p-5 and 8p-6, which are tracked by the monocular SLAM function, are indicated by (p5, R5) and (p6, R6) in the map coordinate system 3s , respectively.”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5, and a frame acquired at the location 8p-6 is a key frame 7kf6 . Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”) and second initial marker information indicating a position and an orientation of a second real marker installed at the position of the target object, and (Jiang Fig. 11, “[0098] When the target 7t is indicated by the instructor 1 in the visual field 7d-5, as described above, the operator terminal 201 specifies the anchor feature point 3pa , and recognizes the target 7t by multiple feature points 3p including the anchor feature point 3pa.”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5, and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt) of the anchor feature point 3pa is specified by using the key frame 7kf-5 for the location 8p-5…”; “[0108] …coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c …”) the spatial calibration includes installing a first virtual marker corresponding to the first real marker in the virtual space based on the first initial marker information, (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”) installing a second virtual marker corresponding to the second real marker in the virtual space so as to maintain a relative positional relationship between the first real marker and the second real marker based on the second initial marker information, and…at the position of the second real marker (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”; “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”; “[0124] The guidance display control part 280 selects one of an option of displaying the instruction detail 2g and another option of the guidance display 9a-6 or 9a-7 (hereinafter, may be generally called “guidance display 9a”)…”) However, Jiang is silent about installing the 3D model. Katsumata teaches installing the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang and Katsumata are analogous art as both of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by installing the 3D model as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Regarding claim 10, Jiang teaches wherein the synchronizing includes determining the position of the first real marker as the position of the virtual camera, and determining the orientation of the first real marker as the orientation of the virtual camera (Jiang Fig. 1, “[0037-0039] The display device 21d may be a device such as the HMD , which is wearable and capable of input and output voice… The camera 21c may be a device such as a head mounted camera (HMC)…”; “[0061-0062] By the monocular SLAM function, information of the position and the posture of the camera 21 (hereinafter, called “position-and-posture information 3c”…”) Regarding claim 11, Jiang teaches wherein the virtual camera video further changes in synchronization with a change in the position and the orientation of the virtual camera (Jiang “[0079] …the operator 1 operates the input device 114 and points to the target 7t on the 3D panorama image 4 displayed at the display device 115 of the remote Support apparatus 101 . The instruction detail 2g is displayed at a precise position with respect to the target 7t at the display device 21d …”; “[0031] The operator 2 at the work site possesses an operator terminal 201, and wears a display device 21d and a camera 21c …”). The Examiner would like to note that the operator 2 is wearing the camera and therefore as they move—there is a change in position and orientation of the virtual camera. Regarding claim 14, Katsumata teaches wherein the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) However, Katsumata is silent about includes a body and a movable portion attached to the body, and the method further comprises acquiring, from the manager terminal, operation information indicating an operation from the manager for the movable portion, and an image indicating the movable portion included in the virtual camera video changes based on the operation information. Jiang teaches includes a body and a movable portion attached to the body, and (Jiang Fig. 1: 7t target , “[0118] The remote support process part 142 of the remote Support apparatus 101 generates the 3D panorama image 4 representing the work environment view from the operator 2…”) the method further comprises acquiring, from the manager terminal, operation information indicating an operation from the manager for the movable portion, and (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4. The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c, to the operator terminal 201 of the operator 2…The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) an image indicating the movable portion included in the virtual camera video changes based on the operation information (Jiang Fig. 1, “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201 . When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2 . Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) Jiang and Katsumata are analogous art as both of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by a 3D model as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Regarding claim 15, Jiang teaches wherein the initial marker information and the real marker information are information acquired from a sensor that allows detection of the position and the orientation of the real marker (Jiang “[0104] A frame acquired at the location 8p-5 is a key frame 7kf-5 , and a frame acquired at the location 8p-6 is a key frame 7kf6. Coordinates (pt, Rt)...”; “[0108] As described in FIG.9, coordinates (p6, R6) of the location 8p-6 are acquired based on the position-and-posture information 3c. ..”; “[0062] By acquiring the position-and-posture information 3c and the feature point map 3m, it is possible to generate an environment map representing a real environment in the three dimensions. A self-position estimation is conducted by the monocular SLAM function based on world coordinate systems. ”; “[0102] The map coordinate system 3s of the feature map 3m represents a coordinate system of a real environment (that is, a coordinate system in a three dimensional real space )…”) Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable and obvious over Jiang and Katsumata as applied to claims 1-11 and 14-17 above, and further in view of US Patent Application Publication US 2020/0391882 A1, (Terry et al.) (hereinafter “Terry”). Regarding claim 12, Terry teaches wherein the output image further includes an overhead view image … and a camera icon displayed in association with the overhead view image, and (Terry “[0086] FIG. 3 is a screen diagram of a view of a graphical user interface ( GUI ) 60 on a mobile device 100 while piloting the UAV 50. The GUI 60 provides a real - time view to the engineer / technician piloting the UAV 50. That is , a screen 62 provides a view from a camera on the UAV 50 . As shown in FIG . 3 , the cell site 10 is shown with the cell site components 14 in the view of the screen 62. Also , the GUI 60 has various controls 64 , 66 …”) a display mode of the camera icon is (Terry “[0086] FIG. 3 is a screen diagram of a view of a graphical user interface ( GUI ) 60 on a mobile device 100 while piloting the UAV 50. The GUI 60 provides a real - time view to the engineer / technician piloting the UAV 50. That is , a screen 62 provides a view from a camera on the UAV 50. As shown in FIG . 3 , the cell site 10 is shown with the cell site components 14 in the view of the screen 62. Also , the GUI 60 has various controls 64 , 66 …”) However, Katsumata and Terry are silent about changed in synchronization with the position and the orientation of the virtual camera. Jiang teaches changed in synchronization with the position and the orientation of the virtual camera (Jiang “[0061] In the operator terminal 201, a monocular Simultaneous Localization And Mapping (SLAM) function…By the monocular SLAM function, information of the position and the posture of the camera 21 (hereinafter, called “position-and-posture information 3c (FIG. 3)) …are simultaneously estimated from the image captured by the camera 21 …”; “[0062] By acquiring the position-and-posture information 3c …it is possible to generate an environment map representing a real environment in the three dimensions. A self-position estimation is conducted by the monocular SLAM function based on world coordinate systems .”; “[0069] …The camera parameter K is generally estimated by using a calibration chart . Alternatively, the skew s=0 and the aspect ratio a-1 may be defined, and the c, and c, may indicate an image center.”) However, Jiang and Terry are silent about the 3D model. Katsumata teaches of the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang, Katsumata, and Terry are analogous art as all of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang by a 3D model as taught by Katsumata and by wherein the output image further includes an overhead view image…and a camera icon displayed in association with the overhead view image, and a display mode of the camera icon is as taught by Terry and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Regarding claim 13, Jiang teaches wherein the output image is displayed on a worker terminal of the worker and a manager terminal of a manager, and (Jiang Fig. 1: “3D Panorama Image 4” and “Image Frame 2c”) the method further comprises: acquiring, from the manager terminal, display object information for displaying a display object indicating a position designated by the manager in the overhead view image or … indicated in the virtual camera video; and (Jiang Fig. 1, “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4 . The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c , to the operator terminal 201 of the operator 2 …The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) displaying the display object to be superimposed on the overhead view image or the virtual camera video based on the display object information (Jiang Fig. 1, “[0033] …A range depicted by an image of the image frame 2c corresponds to the visual field 7d. In the following, the visual field 7d is described as a range where the image of a latest image frame 2c is depicted.”; “[0079] …A method for acquiring a display position of the instruction detail 2g will be described for a case in which the target 7t is inside the visual field 7d and a case in which the target 7t is outside the visual field 7d.”; “[0035-0036] The instructor 1 may click a location desired to indicate in the 3D panorama image 4 . The remote Support apparatus 101 sends the instruction information 2f including location information clicked by the instructor 1, an instruction detail 2g , and the like in the image frame 2c , to the operator terminal 201 of the operator 2 …The display device 21d displays the instruction detail 2g by the visual annotation based on the instruction information 2f which the operator terminal 201 has received.”; “[0041] When the instructor 1 inputs the instruction detail 2g on the image frame 2c displayed at the remote Support apparatus 101, the instruction information 2f is sent to the operator terminal 201. When receiving the instruction information 2f the operator terminal 201 displays the instruction detail 2g at the indicated location specified by the instruction information 2f …”; “[0045] Also, in a case in which the instructor 1 indicates the target 7t in the 3D panorama image 4, it is possible to precisely specify a position of the target 7t with respect to a current head location and the view direction of the operator 2. Accordingly, it is possible to precisely display the instruction detail 2g at the display device 21d of the operator 2 .”) However, Jiang and Terry are silent about the 3D model. Katsumata teaches the 3D model (Katsumata “[0038] …virtual object. In the present exemplary embodiment, the data configuration of the virtual object includes position and orientation information 302 indicating the position ( x , y , and z coordinates ) and orientation ( rolling , pitching , and yawing ) of the virtual object in the three - dimensional space , and model information 304 indicating visual information such as a color and shape of the virtual object…”) Jiang, Katsumata, and Terry are analogous art as all of them are related to target object representation. Therefore, it would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Jiang modified by Terry by a 3D model as taught by Katsumata and use that within Jiang’s operator support system. The motivation for the above is for improved remote support for real-world object control. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMELIA VELAZQUEZ VALENCIA whose telephone number is (571)272-7418. The examiner can normally be reached M-F, 8:30AM-5:00PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A. Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /A.V.V/Examiner, Art Unit 2612 /Said Broome/Supervisory Patent Examiner, Art Unit 2612 Date: 5/26/2026 Application/Control Number: 18/973,716 Page 2 Art Unit: 2612 Application/Control Number: 18/973,716 Page 3 Art Unit: 2612 Application/Control Number: 18/973,716 Page 4 Art Unit: 2612 Application/Control Number: 18/973,716 Page 5 Art Unit: 2612 Application/Control Number: 18/973,716 Page 6 Art Unit: 2612 Application/Control Number: 18/973,716 Page 7 Art Unit: 2612 Application/Control Number: 18/973,716 Page 8 Art Unit: 2612 Application/Control Number: 18/973,716 Page 9 Art Unit: 2612 Application/Control Number: 18/973,716 Page 10 Art Unit: 2612 Application/Control Number: 18/973,716 Page 11 Art Unit: 2612 Application/Control Number: 18/973,716 Page 12 Art Unit: 2612 Application/Control Number: 18/973,716 Page 13 Art Unit: 2612 Application/Control Number: 18/973,716 Page 14 Art Unit: 2612 Application/Control Number: 18/973,716 Page 15 Art Unit: 2612 Application/Control Number: 18/973,716 Page 16 Art Unit: 2612 Application/Control Number: 18/973,716 Page 17 Art Unit: 2612 Application/Control Number: 18/973,716 Page 18 Art Unit: 2612 Application/Control Number: 18/973,716 Page 19 Art Unit: 2612 Application/Control Number: 18/973,716 Page 20 Art Unit: 2612 Application/Control Number: 18/973,716 Page 21 Art Unit: 2612 Application/Control Number: 18/973,716 Page 22 Art Unit: 2612 Application/Control Number: 18/973,716 Page 23 Art Unit: 2612 Application/Control Number: 18/973,716 Page 24 Art Unit: 2612 Application/Control Number: 18/973,716 Page 25 Art Unit: 2612 Application/Control Number: 18/973,716 Page 26 Art Unit: 2612 Application/Control Number: 18/973,716 Page 27 Art Unit: 2612 Application/Control Number: 18/973,716 Page 28 Art Unit: 2612 Application/Control Number: 18/973,716 Page 29 Art Unit: 2612 Application/Control Number: 18/973,716 Page 30 Art Unit: 2612 Application/Control Number: 18/973,716 Page 31 Art Unit: 2612 Application/Control Number: 18/973,716 Page 32 Art Unit: 2612
Prosecution Insights