CTNF 18/985,585 CTNF 90856 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority 02-27 AIA Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP2024-000971 , filed on 2024.01.09 . Should applicant desire to obtain the benefit of foreign priority under 35 U.S.C. 119(a)-(d) prior to declaration of an interference, a certified English translation of the foreign application must be submitted in reply to this action. 37 CFR 41.154(b) and 41.202(e) . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim(s) 1- 21 is/a re rejected under 35 U.S.C. 103 as being unpatentable over Ishi kawa (US 20120069006 A1) in view of Rigau et al. (US 20220083055 A1, hereinafter Rigau). Rega rding Claim 1, Ishikawa teaches an information processing device comprising one or more processors and/or circuitry configured to: (Ishikawa, Paragraph [0037], "The position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction"), [[ execute acquisition processing of acquiring a real image in which real space is image-captured ]] execute generating processing of generating, based on data of a virtual object placed in virtual space, a virtual image representing the virtual space in which the virtual object is placed (Ishikawa, Paragraph [0036], "The display controller 111 superimposes the operation panel image, which is the second image, on the stereoscopic content image, which is the first image containing an object displayed with depth"), execute combining processing of combining the real image and the virtual image, and generating a combined image, and (Ishikawa, Paragraph [0036], "The display controller 111 superimposes the operation panel image... on the stereoscopic content image... to cause the display apparatus 200 to display the superimposed image stereoscopically (that being, a first stereoscopic image) on the display screen thereof."). execute control processing in which, based on whether or not a real object included in the real image and the virtual object included in the virtual image overlap, framing control is performed to adjust positions of the real object and the virtual object in the combined image," (Ishikawa, Paragraph [0036], "The position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction so that overlapping of the object contained in the stereoscopic content image with the operation panel image in the depth direction is reduced."), wherein, in the combining processing, the real image that is image-captured based on the framing control and the virtual image that is generated based on the framing control are combined (Ishikawa, Paragraph [0050], "the display controller 111 of the information processing apparatus 100 has the operation panel image 600 displayed by being superimposed on the stereoscopic content image 500 in which the position in the depth direction is changed in step S107 (step S109)."), But Ishikawa does not explicitly disclose execute acquisition processing of acquiring a real image in which real space is image-captured. However, Rigau further teaches an information processing device comprising one or more processors and/or circuitry configured to: (Rigau, Paragraph [0057], "The system 120 for example comprises a processing device (PROCESSING DEVICE) 202 implemented by one or more networked computers") execute acquisition processing of acquiring a real image in which real space is image-captured , (Rigau, Paragraph [0059], "The computing system 120 for example comprises a fixed camera(s) interface (FIXED CAMERA(S) INTERFACE) 214 that receives raw video streams from the fixed cameras 114 via a wireless or wired interface and transmits these raw video streams to the processing device 202."), execute generating processing of generating, based on data of a virtual object placed in virtual space, a virtual image representing the virtual space in which the virtual object is placed, (Rigau, Paragraph [0070], "The mixed reality module 302 constructs and maintains the virtual world, which is composed of all the virtual elements including the virtual replicas of the robots and the static/mobile real objects"), execute control processing in which, based on whether or not a real object included in the real image and the virtual object included in the virtual image overlap, framing control is performed to adjust positions of the real object and the virtual object in the combined image (Rigau, Paragraph [0059], "the processing device 202 may also generate control signals for controlling the pan, tilt and/or zoom of the fixed camera(s) 114 and/or the robot camera(s) 116."), Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that combine real-world imagery with virtual objects and manage their spatial relationship in a merged display. Ishikawa teaches detecting object overlap and adjusting display positions to reduce it. Rigau teaches camera control (pan, tilt, zoom), which constitutes framing control. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate framing control (e.g., zoom/pan) taught by Rigau into modified invention of Ishikawa such that overlap management system to adjust the position of the captured real object relative to the virtual object to resolve the overlap.). Regarding Claim 2, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the control processing, in a case in which the real object and the virtual object do not overlap, the framing control is performed such that the real object and the virtual object fit in the combined image (Ishikawa, Paragraph [0036], "The position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction so that overlapping... is reduced.", Rigau, Paragraph [0059], "the processing device 202 may also generate control signals for controlling the pan, tilt and/or zoom of the fixed camera(s)", Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that combine real-world imagery with virtual objects and manage their spatial relationship in a merged display. Ishikawa provided a way of detecting overlap between real and virtual objects in the depth direction and adjusting display positions to reduce that overlap. Rigau provided a way of controlling pan, tilt, and zoom of fixed cameras to reframe the captured scene relative to virtual elements. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate camera-based framing control (pan/tilt/zoom) taught by Rigau into the modified invention of Ishikawa such that when the real object and the virtual object do not overlap, the camera framing is adjusted to ensure both objects fit within the combined image, maintaining a usable and compositionally sound display. The motivation is to maintain proper framing of scene elements in a mixed reality composition. Regarding Claim 3, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the control processing, in a case in which a second real object that is different from the real object and a second virtual object that is different from the virtual object overlap, the framing control is performed such that the second real object and the second virtual object are out of an angle of view of the combined image (Ishikawa, Paragraph [0037], "The position changing unit 112 changes the position... so that overlapping... in the depth direction is reduced.", Rigau, "The engine 305 also for example implements anti-collision routines in order to prevent collisions... between any robot and a virtual element") Ishikawa teaches moving objects to resolve overlap. Rigau teaches routines to prevent collisions between real and virtual elements. It would be obvious to incorporate Rigau's anti-collision logic into Ishikawa to adjust the framing (e.g., pan/zoom) such that secondary colliding objects are moved outside the field of view to clear the overlap.) Regarding Claim 4, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the control processing, [[ prediction is performed regarding whether or not a second real object that is different from the real object and a second virtual object that is different from the virtual object move and collide, and ]] in a case of prediction that the second real object and the second virtual object collide, the framing control is performed such that a position at which the second real object and the second virtual object will collide is out of an angle of view of the combined image (Ishikawa, Paragraph [0037], "position changing unit 112 changes the position... so that overlapping... is reduced."). But Ishikawa does not explicitly disclose prediction is performed regarding whether or not a second real object that is different from the real object and a second virtual object that is different from the virtual object move and collide. However, Rigau teaches prediction is performed regarding whether or not a second real object that is different from the real object and a second virtual object that is different from the virtual object move and collide, and" (Rigau, Paragraph [0106], "When the drone 802 reaches a point at a given distance from the wall 902′, the real-virtual interaction engine 305 for example simulates a collision by applying maximum reverse thrust") in a case of prediction that the second real object and the second virtual object collide, the framing control is performed such that a position at which the second real object and the second virtual object will collide is out of an angle of view of the combined image (Rigau, Paragraph [0059], "the processing device 202 may also generate control signals for controlling the pan, tilt and/or zoom"). Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that track the spatial relationship between real and virtual objects and apply control logic to prevent or mitigate harmful overlaps or collisions. Ishikawa provided a way of detecting current overlap between objects and repositioning them in the depth direction to reduce overlap. Rigau provided a way of predictively monitoring moving real objects (e.g., drones) relative to virtual boundary elements (e.g., virtual walls) to simulate and anticipate future collisions before they occur. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate predictive collision detection of a second real object and a second virtual object taught by Rigau into the modified invention of Ishikawa such that when a predicted future collision is identified, Ishikawa's repositioning/framing logic (via Rigau's camera control) is applied to place the predicted collision point outside the angle of view of the combined image. The motivation is to proactively prevent visual disruption from anticipated collisions between secondary real and virtual objects. Regarding Claim 5 , the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the control processing, the framing control is performed for each of the real object and the virtual object." (Ishikawa, Paragraph [0057], "The position changing unit 112... may change only the position of the object 510 in the depth direction. In such a case, however... the position changing unit 112 may change the positions of other objects such as the object 520 and the object 530 in the depth direction"). Regarding Claim 6 , the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches in the control processing, further, a first timing for acquiring the real image and a second timing for generating the virtual image are specified based on whether or not the real object and the virtual object overlap, and (Rigau, [0006], "events involving virtual elements in a virtual world can be synchronized with the dynamic behavior of real objects in the physical world.", [0072], "virtual features generated by the real-virtual interaction engine 305 are synchronized in time and space and merged with the raw video stream(s)"). As explained in the rejection of Claim 1, the obviousness for combining the camera acquisition and framing control of Rigau into Ishikawa is provided above. Regarding Claim 7, Ishikawa teaches an information processing device comprising one or more processors and/or circuitry configured to: (Ishikawa, Paragraph [0037], "The position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction"), [[ execute acquisition processing of acquiring a real image in which real space is image-captured ]] execute generating processing of generating, based on data of a virtual object placed in virtual space, a virtual image representing the virtual space in which the virtual object is placed (Ishikawa, Paragraph [0036], "The display controller 111 superimposes the operation panel image, which is the second image, on the stereoscopic content image, which is the first image containing an object displayed with depth") execute control processing of, based on whether or not a real object included in the real image and the virtual object included in the virtual image overlap (Ishikawa, Paragraph [0036], "The position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction so that overlapping of the object contained in the stereoscopic content image with the operation panel image in the depth direction is reduced."), [[ controlling a first timing of acquiring the real image and a second timing for generating the virtual image ]], and . execute combining processing of combining the real image acquired at the first timing and the virtual image generated at the second timing and generating a combined image (Ishikawa, Paragraph [0036], "superimposes the operation panel image... on the stereoscopic content image"). But Ishikawa does not explicitly disclose execute acquisition processing of acquiring a real image in which real space is image-captured. execute combining processing of combining the real image acquired at the first timing and the virtual image generated at the second timing and generating a combined image. However, Rigau teaches execute acquisition processing of acquiring a real image in which real space is image-captured (Rigau, Paragraph [0059], "The computing system 120 for example comprises a fixed camera(s) interface (FIXED CAMERA(S) INTERFACE) 214 that receives raw video streams from the fixed cameras 114 via a wireless or wired interface and transmits these raw video streams to the processing device 202."), execute generating processing of generating, based on data of a virtual object placed in virtual space, a virtual image representing the virtual space in which the virtual object is placed, (Rigau, Paragraph [0070], "The mixed reality module 302 constructs and maintains the virtual world, which is composed of all the virtual elements including the virtual replicas of the robots and the static/mobile real objects"), execute combining processing of combining the real image acquired at the first timing and the virtual image generated at the second timing and generating a combined image (Rigau, Paragraph [0072], "synchronized in time and space and merged with the raw video stream(s)”), execute control processing of, based on whether or not a real object included in the real image and the virtual object included in the virtual image overlap, controlling a first timing of acquiring the real image and a second timing for generating the virtual image, and (Rigau, Paragraph [0072], "virtual features generated by the real-virtual interaction engine 305 are synchronized in time and space and merged with the raw video stream(s).", Rigau, Paragraph [0071], "The engine 305 also for example implements anti-collision routines in order to prevent collisions between robots themselves and/or between any robot and another real object... and in some cases between any robot and a virtual element"). Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that combine real-world imagery with virtual objects and manage their spatial relationship in a merged display. Ishikawa teaches the detection and mitigation of object overlap. Rigau teaches temporal synchronization and anti-collision logic to prevent real-to-virtual collisions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Rigau’s synchronization and anti-collision logic into Ishikawa's system to adjust the specific capture and generation timings (first and second timing) to ensure that the objects do not overlap/collide in the merged video stream.) Regarding Claim 8, the combination of Ishikawa and Rigau teaches the invention in Claim 6. The combination further teaches wherein, in the control processing, a position of the virtual object after a predetermined amount of time is predicted based on movement information of the virtual object that is included in the data of the virtual object, and whether or not the real object and the virtual object placed at the predicted position overlap is determined ." (Rigau, Paragraph [0106], "When the drone 802 reaches a point at a given distance from the wall 902′, the real-virtual interaction engine 305 for example simulates a collision"; [0070], “the real-virtual interaction engine 305 receives the tracking data TRACKING DATA from the tracking system 112 and uses the data stored in the database 306 to ensure synchronization of the 6 DoF coordinates… between the real elements… and their corresponding virtual replicas in the virtual world"; [0066], “robot data, including at least for each robot, a 3D model and a dynamic model respectively indicating the 3D shape and the dynamic behavior of the robot”). Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that monitor the spatial and temporal relationship of real and virtual objects to prevent overlap or collision. Ishikawa provided a way of detecting current spatial overlap between real and virtual objects in the depth direction and applying corrective repositioning. Rigau provided a way of monitoring movement information (speed and distance) of a moving real object relative to a virtual object to predict a future collision at a specific position. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate movement-based predictive position determination taught by Rigau into the modified invention of Ishikawa such that the system predicts the future position of the virtual object based on its movement data and proactively determines whether the real and virtual object will overlap at that predicted position. The motivation is to anticipate and prevent future overlaps before they occur by leveraging object movement information. Regarding Claim 9, the combination of Ishikawa and Rigau teaches the invention in Claim 8. The combination further teaches in the control processing, in a case in which the real object and the virtual object overlap (Ishikawa, Paragraph [0041\, "the object 510 overlaps with the operation panel image 600 displayed on the reference plane matching the display screen 210 in the depth direction. Therefore, it becomes difficult for the user to visually recognize a portion of the operation panel image 600 where the object 510 overlaps so that, for example, the operation of the information processing apparatus 100 using the operation panel image 600 is hindered ." ; [0037], "the position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction so that overlapping of the object contained in the stereoscopic content image with the operation panel image in the depth direction is reduced”]. Ishikawa does not explicitly disclose but Rigau teaches a position of the virtual object after the predetermined amount of time is predicted. (Rigau, Paragraph [0106], "When the drone 802 reaches a point at a given distance from the wall 902’, the real-virtual interaction engine 305 for example simulates a collision by applying maximum reverse thrust CMDMAX to the drone 802 to simulate a rebound from the wall 902."; it is noted that the system tracks a moving object's approach trajectory toward a virtual element and, upon reaching a defined distance threshold (i.e., after a predetermined amount of time), determines the future spatial position of that object relative to the virtual boundary — constituting a prediction of the virtual object's position after a predetermined amount of time). Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that monitor the spatial and temporal relationship of real and virtual objects to prevent overlap or collision. Ishikawa provided a way of detecting a current overlap between a real object and a virtual object in the depth direction and applying corrective position changes to reduce that overlap. Rigau provided a way of continuously tracking the 6 DoF coordinates and dynamic behavior of moving objects using robot dynamic models and, upon detection that two objects are on an overlapping/collision course, predicting the future position of the moving object after a predetermined amount of time (the distance threshold) to preemptively apply a corrective response before the overlap physically occurs. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Rigau's movement-based predictive position tracking into the modified invention of Ishikawa such that, in a case in which the real object and the virtual object overlap (as detected by Ishikawa's overlap detection), the system predicts the future position of the virtual object after a predetermined amount of time using movement information included in the virtual object's data — allowing the system to identify whether the overlap will persist or resolve and to select an appropriate future timing for image combination. The motivation is to enable proactive, forward-looking management of overlap conditions by leveraging continuous object tracking and dynamic models — rather than only reacting to current overlap — in order to improve the reliability and quality of the combined image output. Regarding Claim 10, the combination of Ishikawa and Rigau teaches the invention in Claim 8. The combination further teaches wherein, in the control processing, in a case in which the real object and the virtual object placed at the predicted position do not overlap, a timing after the predetermined amount of time is specified as the second timing." (Rigau, Paragraph [0072], "virtual features generated by the real-virtual interaction engine 305 are synchronized in time and space and merged with the raw video stream(s)"; it is noted Rigau teaches temporal synchronization. It would be obvious to use the predicted non-overlapping future time as the specified generation timing (second timing) for the virtual object.). As explained in the rejection of Claim 8, the obviousness for combining predictive synchronization of Rigau into Ishikawa is provided above. Regarding Claim 11, the combination of Ishikawa and Rigau teaches the invention in Claim 8. The combination further teaches wherein, in the combining processing in a case in which the real object and the virtual object are overlapped even after a prediction period that is set in advance elapses from a clock time of acquiring the real image (Ishikawa, Paragraph [0072], “If, for example, the position changing unit 112 changes the position of the operation panel image 600 in the depth direction and even thereafter, for example, the object 510 and the operation panel image 600 overlap, like the first exemplary embodiment or the second exemplary embodiment, the position of the object 510 in the depth direction may be changed”; "if, for example, the object 510 and the operation panel image 600 overlap even after an allowable range of a predetermined width is set to the position of the operation panel image 600 in the depth direction after the position change and the position of the operation panel image 600 in the depth direction is changed toward the forward side within the allowable range”). However, Ishikawa does not explicitly disclose that a prediction period that is set in advance elapses from a clock time of acquiring the real image. But, Rigau teaches a prediction period that is set in advance elapses from a clock time of acquiring the real image (Rigau, Paragraph [0013], "the instructions further cause the one or more processing cores to continuously track the 6 Degrees of Freedom coordinates of the first robot corresponding to its position and orientation based on tracking data provided by a tracking system”; it is noted Each tracking cycle corresponds to acquiring a real image at a clock time, with the next tracking cycle defining the prediction period set in advance from that clock time of acquiring the real image). the real image and the virtual image are not combined (Ragu, Paragraph [0106], "the real-virtual interaction engine 305 for example simulates a collision by applying maximum reverse thrust CMDMAX to the drone 802 to simulate a rebound from the wall 902′. In response, the drone 802 for example slows rapidly to a halt, and then starts reversing, for example without ever passing the virtual wall 902’”; it is noted because the real object never passes into the virtual element's spatial position, no image in which the real image and the virtual image are combined at that overlapping position is ever generated i.e., the real image and the virtual image are not combined). As explained in the rejection of Claim 8, the obviousness for combining Rigau into Ishikawa is provided above. Regarding Claim 12, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the combining processing, in a case in which the real object and the virtual object overlap, the real image and the virtual image are not combined." ( Ishikawa, Paragraph [0041], "Therefore, it becomes difficult for the user to visually recognize a portion... operation... is hindered."; it is noted overlap makes content unusable. It would be obvious to a person of ordinary skill in the art to choose not to combine (forgo display) if the overlap cannot be resolved). Regarding Claim 13, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the control processing, whether or not the real object and the virtual object overlap in a two-dimensional plane of the combined image is determined (Ishikawa, Paragraph [0055], "the object 510 overlaps, in the depth direction, with the operation panel image 600 displayed on the reference plane matching the display screen 210."; it is noted determining overlap relative to a reference plane (the display screen), which is a 2D surface.). Regarding Claim 14, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches whether or not the real object and the virtual object overlap in a three-dimensional space (Ishikawa, Paragraph [0037], “the position changing unit 112 changes the position of the stereoscopic content image, object, or operation panel image in the depth direction so that overlapping of the object contained in the stereoscopic content image with the operation panel image in the depth direction is reduced."; [0061], "the Z coordinate information 703 is information about the position in the depth direction of each object contained in the stereoscopic content image 500."; it is noted determines overlap using three-dimensional coordinate information, including the depth (Z-axis) direction). Ishikawa does not explicitly disclose but Rigau teaches in which the real space and the virtual space are merged (RIgau, Paragraph [0005], ““Mixed reality”, also known as hybrid reality, is the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects can coexist and interact in real-time”; [0070], "the real-virtual interaction engine 305 receives the tracking data TRACKING DATA from the tracking system 112 and uses the data stored in the database 306 to ensure synchronization of the 6 DoF coordinates position and orientation between the real elements… and their corresponding virtual replicas in the virtual world."; [0071], "The engine 305 also for example implements anti-collision routines in order to prevent collisions between robots themselves and/or between any robot and another real object in the activity zone 102, and in some cases between any robot and a virtual element in the virtual world"). Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that track the positions of real and virtual objects in a shared three-dimensional space and determine whether those objects spatially overlap. Ishikawa provided a way of determining overlap between a real content object and a virtual overlay image along the 3D depth (Z-axis) direction using per-object coordinate information. Rigau provided a way of maintaining a merged real-virtual three- dimensional coordinate space in which the 6 DoF positions of real objects and their virtual replicas are continuously synchronized, and determining whether real and virtual objects overlap within that unified space. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Rigau's merged real-virtual three-dimensional coordinate space into the modified invention of Ishikawa such that whether or not the real object and the virtual object overlap in a three-dimensional space in which the real space and the virtual space are merged, is determined.The motivation is to enable accurate real-to-virtual overlap detection using co-registered 3D spatial coordinates shared by both real and virtual objects Regarding Claim 15, the combination of Ishikawa and Rigau teaches the invention in Claim 1. The combination further teaches wherein, in the control processing, coordinates of the virtual object in the three-dimensional space are acquired based on coordinates of the real object in the three-dimensional space." (Rigau, Paragraph [0008], "maintain a virtual world involving at least a first virtual replica corresponding to a first robot in the real world"). Rigau and Ishikawa are analogous since both of them are dealing with mixed reality systems that maintain and track the positions of real and virtual objects in a shared three-dimensional space. Ishikawa provided a way of managing object overlap in three-dimensional depth space by tracking real and virtual object positions along the depth axis. Rigau provided a way of maintaining a virtual world of virtual replicas whose positions are mapped directly to tracked real-world object coordinates. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate coordinate mapping of virtual objects based on real object coordinates taught by Rigau into the modified invention of Ishikawa such that the coordinates of the virtual object in three-dimensional space are derived from the known coordinates of the corresponding real object, enabling accurate spatial overlap determination. The motivation is to anchor virtual object positions to real-world spatial coordinates for consistent and accurate 3D overlap detection. Regarding Claim 16, the combination of Ishikawa and Rigau teaches the invention in Claim 15. The combination further teaches wherein, in the control processing, the coordinates of the real object in the three-dimensional space are acquired based on a position and attitude of an image-capturing device that performs image-capturing of the real space, and a distance from the image-capturing device to the real object ." (Rigau, Paragraph [0059], "the processing device 202 may also generate control signals for controlling the pan, tilt and/or zoom of the fixed camera(s)", [0097],"calculated as the difference between the desired robot state and the measured robot state... based on the tracking data", As explained in the rejection of Claim 15, the obviousness for combining 3D coordinate mapping of Rigau into Ishikawa is provided above. Regarding Claim 16, the combination of Ishikawa and Rigau teaches the invention in Claim 15. The combination further teaches wherein clock time in the virtual space is synchronized with clock time in the real space (Rigau,Paragraph [0006], "events involving virtual elements in a virtual world can be synchronized with the dynamic behavior of real objects in the physical world."). Rigau and Ishikawa are analogous since both of them are dealing with mixed/augmented reality image processing systems that combine virtual content with real-world content while controlling positional relationships and interactions between virtual and real elements. Ishikawa provided a way of reducing overlap/interference between displayed contents in a stereoscopic environment by adjusting positions of virtual and real image contents to improve image presentation and user perception. Rigau provided a way of synchronizing events and behaviors between virtual elements in a virtual world and real objects in the physical world, including synchronization in time and space between virtual features and real-world video streams. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate synchronization of clock time between the virtual space and the real space taught by Rigau into the modified invention of Ishikawa such that virtual events, virtual objects, and real-world objects/images in the combined mixed reality environment are temporally synchronized during presentation and interaction processing. The motivation is to improve consistency and realism between the virtual environment and the real-world environment, thereby allowing virtual features and realworld behaviors to operate coherently and in synchronization during mixed reality image generation and interaction processing, as discussed by Rigau in Paragraph [0006]. Regarding Claim 18, it recites limitations similar in scope to the limitations of Claim 1 but as a method and the combination of Ishikawa and Rigau teaches all the limitations as of Claim 1. Therefore is rejected under the same rationale. Regarding Claim 19, it recites limitations similar in scope to the limitations of Claim 7 but as a method and the combination of Ishikawa and Rigau teaches all the limitations as of Claim 7. Therefore is rejected under the same rationale. Regarding Claim 20, it recites limitations similar in scope to the limitations of claim 1 and the combination of Ishikawa and Rigau teaches all the limitations as of Claim 1. And Ishikawa discloses these features can be implemented on a computer-readable storage medium (Ishikawa, Paragraph [0008], “Consistent with a further exemplary embodiment, a non transitory, computer-readable storage medium stores a program that, when executed by a processor, causes the processor to perform a method for processing information”). Regarding Claim 21, it recites limitations similar in scope to the limitations of claim 7 and the combination of Ishikawa and Rigau teaches all the limitations as of Claim 7. And Ishikawa discloses these features can be implemented on a computer-readable storage medium (Ishikawa, Paragraph [0008], “Consistent with a further exemplary embodiment, a non transitory, computer-readable storage medium stores a program that, when executed by a processor, causes the processor to perform a method for processing information”) . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 11577159 B2 Realistic virtual/augmented/mixed reality viewing and interactions US 10896543 B2 Methods and systems for augmented reality to display virtual representations of robotic device actions US 10546424 B2 Layered content delivery for virtual and augmented reality experiences US 20170358024 A1 VIRTUAL REALITY SHOPPING SYSTEMS AND METHODS US 9836117 B2 Autonomous drones for tactile feedback in immersive virtual reality US 20170243403 A1 REAL-TIME SHARED AUGMENTED REALITY EXPERIENCE US 20160267720 A1 Pleasant and Realistic Virtual/Augmented/Mixed Reality Experience Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /YuJang Tswei/Primary Examiner, Art Unit 2614 Application/Control Number: 18/985,585 Page 2 Art Unit: 2614 Application/Control Number: 18/985,585 Page 3 Art Unit: 2614 Application/Control Number: 18/985,585 Page 4 Art Unit: 2614 Application/Control Number: 18/985,585 Page 5 Art Unit: 2614 Application/Control Number: 18/985,585 Page 6 Art Unit: 2614 Application/Control Number: 18/985,585 Page 7 Art Unit: 2614 Application/Control Number: 18/985,585 Page 8 Art Unit: 2614 Application/Control Number: 18/985,585 Page 9 Art Unit: 2614 Application/Control Number: 18/985,585 Page 10 Art Unit: 2614 Application/Control Number: 18/985,585 Page 11 Art Unit: 2614 Application/Control Number: 18/985,585 Page 12 Art Unit: 2614 Application/Control Number: 18/985,585 Page 13 Art Unit: 2614 Application/Control Number: 18/985,585 Page 14 Art Unit: 2614 Application/Control Number: 18/985,585 Page 15 Art Unit: 2614 Application/Control Number: 18/985,585 Page 16 Art Unit: 2614 Application/Control Number: 18/985,585 Page 17 Art Unit: 2614 Application/Control Number: 18/985,585 Page 18 Art Unit: 2614 Application/Control Number: 18/985,585 Page 19 Art Unit: 2614 Application/Control Number: 18/985,585 Page 20 Art Unit: 2614 Application/Control Number: 18/985,585 Page 21 Art Unit: 2614 Application/Control Number: 18/985,585 Page 22 Art Unit: 2614 Application/Control Number: 18/985,585 Page 23 Art Unit: 2614 Application/Control Number: 18/985,585 Page 24 Art Unit: 2614
Prosecution Insights