DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . It is responsive to the submission dated 03/04/2025. Claims 1-20 are presented for examination. Claims 1, 13 and 20 are independent claims.
Information Disclosure Statement
2. The information disclosure statements (IDSs) submitted on 03/04/2025 are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1, 6, 8-10, 13, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Luo (CN107077755 A).
Considering claim 1, Luo discloses a method for adjusting spatial elements (e.g., method and system implementing an adjustable virtual superposition of reality elements that merge the user's surroundings on the virtual content of the virtual reality system. See para. 8), the method comprising:
displaying an environmental picture, the environmental picture including a picture
obtained by image acquisition of a scene region by an extended reality (XR) device (e.g., Luo discloses: acquiring a real three-dimensional panoramic image of a real environment around a user; rendering the real three-dimensional panoramic image according to the rendering virtual degree selected by the user; and outputting the result obtained by rendering to a virtual reality display module. See paras. 11-13 and 16-19, wherein the virtual reality headset, which further comprises a helmet and a reality capturing module corresponds with the extended reality (XR) device. Paragraph 103 of Luo further discloses the electronic device to be mixture of virtual and real environment; i.e., an extended reality (XR) device).
In addition, Luo discloses: using a processor module, coupled to the reality capturing module of the electronic device, for identifying and extracting the real elements in the real three-dimensional panoramic image to acquire a first rendering virtual degree to be used for indicating the proportion of the real elements which the user wants to reserve; and processing the real elements in the real three-dimensional panoramic image according to the acquired first rendering virtual degree. See para. 20. The processor module is used for outputting the rendered result to the virtual reality display module. See paras. 17 and 40, wherein the outcome of the first rendering virtual degree used for processing the extracted real elements in the real three-dimensional panoramic image corresponds to the displayed first spatial element of the scene region.
Furthermore, Luo discloses: The reality capturing module can be a three-dimensional image recognition sensor capable of conducting real-time modeling…. and if a plurality of three-dimensional image recognition sensors is used for splicing the real three-dimensional panoramic image, a space model is constructed, and the head only displays another part in the space when rotating. See para. 35, wherein the space model constructed by the image recognition sensor corresponds to the 3D simulation structure of the scene object in the scene region.
Thus, Luo’s teachings obviously encompas: displaying a first spatial element generated by three-dimensional layout recognition of the scene region based on the environmental picture, the first spatial element representing a three-dimensional simulation structure of a scene object in the scene region, as claimed.
Moreover, Luo discloses: in order to realize the selection of the number of rendering objects, the interaction module includes a number adjustment interaction device, and a user realizes the setting of the first rendering virtual degree through the number adjustment interaction device. See para. 63. The reality element extraction module extracts safety reality elements and determines other reality elements selected by a user; and the image processing module determines fusion rendering information of the safety reality elements and other reality elements selected by the user. See paras. 81-82. The image processing module sorts all the real elements according to the matching fusion degree of the matching fusion module according to the position of the real element number interaction device 40 and the three-dimensional model of the current virtual content, and judges and selects which real elements of other real elements selected by a user can be more fused into the current virtual content from the shape structure model, for example, the three-dimensional model of the real elements in the VR virtual world can be adjusted, for example, the model of a building is modified into a castle, but the peripheral size of the castle is ensured to be basically consistent with the building as much as possible. The matching and fusing module can specifically sort the three-dimensional models of the real elements according to the degree of integration, and select the first plurality of suitable three-dimensional models of the real elements according to the position of the real element number interaction device 40 for rendering. The safety problem of the user needs to be fully considered when selecting, and all elements which possibly threaten personal safety, such as steps, walls, rivers and the like, need to be displayed. And performing surface rendering capable of being merged into the VR virtual three-dimensional scene on each selected real element according to the position of the rendering virtualization degree interaction device 30, the three-dimensional model of the current virtual content and rendering data. If the three-dimensional model of the real elements of a building is modified into the three-dimensional model of the castle with the same bottom outer diameter, the three-dimensional model of the castle is blended into the surface rendering of virtual contents, such as style, color, surface material, freshness, sun reflection and the like. See paras. 83-84. During user movement or head rotation, the central processing module processes the display and rendering of the synchronized virtual content and real elements in the surrounding reality in real time, according to the data of the sensor. And when seeing the virtual world, the user can automatically adjust the number and the degree of virtualization of the real elements displayed in the virtual three-dimensional world by the surrounding real environment. See paras. 86-87.
In short, Luo discloses using the shape structure model to fuse selected other real elements into the current virtual content, so that the three-dimensional model of the real elements in the VR virtual world can be adjusted, while maintaining the peripheral size of the 3D model; and then performing surface rendering capable of being merged into the VR virtual three-dimensional scene on each selected real element according to the position of the rendering virtualization degree interaction device, and display the adjusted other elements in real-time, based on user selection and according to the peripheral size and shape of the structural features from the scene captured of the real environment. Luo’s teachings, therefore, encompass displaying, based on an adjustment operation on the first spatial element, an adjusted spatial layout element, the adjustment operation adjusting the three-dimensional simulation structure of the scene object, the adjusted spatial layout element being used to generate a three-dimensional virtual environment corresponding to the scene region in an XR application.
As such, it is submitted that the Luo reference obviously met all the features of claim 1. And a person skilled in the art, given the teachings of Luo, could have easily used the techniques described in the Luo reference to adjust the layout of spatial elements for simulating the 3D structure of the scene object for used in generating a three-dimensional virtual environment corresponding to the scene region in an XR application. Modifying the Luo reference as such can be performed without exercising inventive skill, as doing so would have been a predictable variation of the teachings in Luo, and would be beneficial for improving the immerse experience of the user. See para. 87 of Luo.
As per claim 6, Luo discloses the XR device includes a first camera and a second camera; and the displaying the environmental picture comprises: obtaining a first image and a second image, the first image and the second image being respectively obtained by the first camera and the second camera performing depth image acquisition on the scene region at the same time (see paras. 35, 37-38, 52, 54-57, 67-70 and 80); and displaying the environmental picture based on an image fusion result corresponding to the first image and the second image (see paras. 39-40 and 56-57, 71-72 and 76-77).
As per claim 8, Luo discloses obtaining a background object in the scene region (e.g., virtual content in virtual 3D scene); and projecting a virtual object (e.g., virtual elements) onto an object plane of the background object to generate the three-dimensional virtual environment corresponding to the scene region. See paras. 8 and 76-77 and 104.
As per claim 9, Luo discloses determining an environmental sound effect corresponding to the three-dimensional virtual environment based on (i) a distance between the scene object and the XR device and (ii) a material corresponding to the scene object. See paras. 57-60, 65, 71-72 and 84-87 and 104.
As per claim 10, Luo discloses the first spatial element indicates a simulation
region parameter corresponding to the scene region, and the adjusted spatial layout
element indicates a modified simulation region parameter adjusted to meet layout requirements. See paras. 81-84 and 104-105.
The subject matters of claims 13 and 20 correspond in terms of an apparatus and computer-readable medium, respectively, to that of independent method claim 1. The features of claims 13 and 20 are substantially the same as those of claim 1 except the invention category. Accordingly, the same reasonings applied for the rejections of claim 1 also apply to claims 13 and 20.
Claim 18 is rejected under the same rationale as claim 18.
Allowable Subject Matter
5. Claims 2-5, 7, 11-12, 14-17 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, because the prior arts of record fail to teach the method and apparatus according to claims 1 and 13, wherein the first spatial element includes a plurality of spatial nodes corresponding to a target scene object in the scene region; and the displaying the adjusted spatial layout element comprises: receiving a node adjustment operation on a first spatial node of the plurality of spatial nodes; determining at least one second spatial node from the plurality of spatial nodes based on the target scene object and the first spatial node, the at least one second spatial node having a positional binding relationship with the first spatial node that synchronizes adjustment of the at least one second spatial node with the first spatial node; adjusting layout positions of the first spatial node and the at least one second spatial node in the scene region based on the node adjustment operation; and displaying the adjusted spatial layout element based on the adjusted layout positions (as recited in claims 2 and 14); wherein the first spatial element includes a plurality
of spatial line segments corresponding to the scene object; and the displaying the adjusted spatial layout element comprises: receiving a line segment adjustment operation on a first spatial line segment of the plurality of spatial line segments;
adjusting the first spatial line segment based on the line segment adjustment operation,
and updating line segment parameters of connected spatial line segments to generate a line segment adjustment result corresponding to the plurality of spatial line segments; and displaying the adjusted spatial layout element based on the line segment adjustment result (as recited in claims 4 and 16); and wherein the first spatial element includes a plurality of spatial planes corresponding to the scene object; and the displaying the adjusted spatial layout element comprises: determining, based on a plane removal operation on a first spatial plane of the plurality of spatial planes is received, candidate spatial planes connected to the first spatial plane; determining, from the candidate spatial planes, a second spatial plane for which a number of plane connections is less than a connection threshold; removing the first spatial plane and the second spatial plane to obtain a plane removal result; and displaying the adjusted spatial layout element based on the plane removal result (as recited in claims 5 and 17); and wherein the XR device is a first XR device worn by a first user; and the displaying the environmental picture comprises: receiving real-time rendering data from a second XR device worn by a second user, the second XR device performing the image acquisition on the scene region; and displaying the environmental picture based on the real-time rendering data (as recited in claim 12).
The prior arts of record fail also to teach the method and apparatus according to claims 6 and 18, wherein the displaying the first spatial element comprises: obtaining target inertial data, the target inertial data including inertial data obtained by the XR device; generating a world coordinate system corresponding to the scene region based on the first image, the second image, and the target inertial data, the world coordinate system including three-dimensional space coordinates corresponding to the first spatial element; performing coordinate conversion on the three-dimensional space coordinates of the first spatial element based on (i) a relative position between the first camera and the second camera and (ii) respective shooting parameters of the first camera and the second camera, to obtain two-dimensional plane coordinates corresponding to the three-dimensional space coordinates of the first spatial element; and rendering the environmental picture based on the two-dimensional plane coordinates to display the first spatial element (as recited in claims 7 and 19); and the method according to claim 10, wherein the simulation region parameter includes at least one of a simulation region area and a simulation region floor height; and the displaying the adjusted spatial layout element comprises: when the adjustment operation on the first spatial element is to reduce the simulation region area, displaying the adjusted spatial layout element based on an area reduction magnitude indicated by the adjustment operation; and when the adjustment operation on the first spatial element is to reduce the simulation region floor height, displaying the adjusted spatial layout element based on a floor height reduction magnitude indicated by the adjustment operation (as recited in claim 11).
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zhang et al. (CN 108564660 A) discloses an interaction method and system of three-dimensional elements in a two-dimensional elements and virtual reality environment, wherein the method comprises: defining a two-dimensional element, and the two-dimensional element number, classifying the three dimensional element of two-dimensional elements in the virtual reality environment for binding; obtaining attribute data and the state data of the two-dimensional element on two-dimensional operating platform by reacTIVision, and uses the TUIO procedure to the attribute data and the state data to analyze the attribute data and the state data after analyzing to the UE4 editor; UE4 editor according to attribute data of the two-dimensional element adjustable three-dimensional virtual reality environment taking according to the state of the corresponding data state adjusting the three-dimensional elements and performing three-dimensional display. In the invention, incorporating virtual reality and augmented reality technology, the 2D elements are completely matched, with the VR scenes by 2D element can be directly positioned to the VR scene, and real time controls the position of VR scenes, faces, plan changes and other information.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to WESNER SAJOUS whose telephone number is (571) 272-7791. The examiner can normally be reached on M-F 10:00 TO 7:30 (ET).
Examiner interviews are available via telephone and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice or email the Examiner directly at wesner.sajous@uspto.gov.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached on 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESNER SAJOUS/Primary Examiner, Art Unit 2612
WS
09/02/2026
Prosecution Insights