DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This office action is responsive to the amendment filed on 07/01/2026.
Claim(s) 1-20 is/are pending in the application.
Independent claim(s) 1, 8, 15 was/were amended.
Response to Arguments
Applicant's argument(s), regarding the amended portion(s) as recited in independent claim 1 (and similarly in independent claim(s) 8, 15), filed 07/01/2026, have/has been fully considered and is/are persuasive. However, upon further consideration, a new ground(s) of rejection is made, adding/using Borgeat and Mattila to be relied upon for the aforementioned amended portion(s). To note, applicant's amendment necessitated the new ground(s) of rejection presented in this office action.
Terminal Disclaimer
The terminal disclaimer filed on 07/01/2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of US Pat. 12,165,270 has been reviewed and is accepted. The terminal disclaimer has been recorded.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-4, 6-11, 13-16, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Borgeat et al. (“Collaborative Visualization and Interaction for Detailed Environment Models”) in view of Mattila (US 2014/0340395 A1).
In regards to claim 1, Borgeat teaches a method comprising:
generating a three-dimensional (3D) object in a virtual environment of a video conference session (e.g. Abstract: DIMENSION, a new adaptive technological platform that allows several distant groups of users to dynamically interact, in a multimedia fashion, around detailed digital 3D models of objects and environments; Section 4, 1st paragraph: users at different locations must be able to directly interact and discuss when exploring the model; for this purpose, DIMENSION integrates an audio and video infrastructure based on the H.323 protocol, which allows interoperability with existing desktop teleconference tools such as NetMeeting and GnomeMeeting; Examiner’s note: where the teleconferences are viewed as video conference sessions), wherein the 3D object includes multiple 3D models at different levels of detail based on how far away users are from the 3D object within the virtual environment (e.g. Section 3, 4th paragraph: Fig.2 shows the displayed geometry of the same model for different positions of the observer and corresponding increase in levels of details as the viewer approaches the wall; Examiner’s note: this shows a generated 3D object in a virtual environment that includes multiple 3D models at different levels of detail based upon how far away the user is from the 3D object);
generating a two-dimensional (2D) data representation of the 3D object (e.g. Fig.6; Examiner’s note: 3D wall/cave object viewed to be rendered in 2D (i.e. 2D data representation)), enabling a first set of users to view and edit a 3D data representation of the 3D object (e.g. Section 1, 2nd paragraph: infrastructure supports modification of the 3D model: a tool for annotating models during multi-user sessions is shown as an example; Section 7, 1st paragraph: Fig.6 shows a video connection of a remote user, as described in Section 4, as well as the model annotation capability through the use of a 3D drawing system that writes directly on the 3D surface located under the cursor, controlled by the mouse or the laser pointer) at a same time as enabling a second set of users to view and edit the 2D data representation of the 3D object (e.g. Section 6, 1st paragraph: since the users collaborating from different physical locations can be alone or in group, it is necessary to allow them to interact in a collaborative manner on the same virtual environment through the use of wall-sized displays; Examiner’s note: Figs.5-6 show multiple users (e.g. first and second group) can edit a view the 2D data representation of the 3D object (e.g. rendering of the 3D object) at the same time by adding/viewing annotations); and
providing the 2D data representation for display in the video conference session (e.g. Section 5, 1st and 2nd paragraphs: where each eyes view is rendered on one computer, and displayed using a digital micromirror device projector; Examiner’s note: shows 2D data representations for the left and right eye viewpoints are provided for display on the wall display for the video conference session),
but does not explicitly teach the method,
wherein generating the 2D data representation comprises attaching 3D data of the 3D object to the 2D data representation, the 3D data including at least a location and a pose of the 3D object to enable reconstruction of the 3D object from the 2D data representation.
However, Mattila teaches a method, comprising:
comprises attaching 3D data of the 3D object to the 2D data representation, the 3D data including at least a location and a pose of the 3D object to enable reconstruction of the 3D object from the 2D data representation (e.g. [0003]: receiving a mobile image file comprising image data of a geographic area; the image may also include orientation data and location data; a virtual image of the three dimensional model as a function of the alternate pose may be generated; the image data and the virtual image may be combined to create a stereoscopic image or a stereo image; the stereoscopic image may provide a view of the area with a three dimensional effect; Examiner’s note: where the 3D stereoscopic image is viewed as a 3D reconstruction of the 3D object).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Borgeat to attach 3D data, in the same conventional manner as taught by Mattila as both deal with stereoscopic display of 3D data. The motivation to combine the two would be that the included orientation and location data would help enable reconstruction 3D object/environment by using the data for positioning.
In regards to medium claim 8 and system claim 15, claim(s) 8, 15 recite(s) limitations that is/are similar in scope to the limitations recited in claim 1. Therefore, claim(s) 8, 15 is/are subject to rejections under the same rationale as applied hereinabove for claim 1. To note, Oz discloses the use of processing circuits and memory in paragraph [0124].
In regards to claim 2, the combination of Borgeat and Mattila teaches a method, wherein the virtual environment comprises a virtual reality (VR) environment including 3D avatars of one or more users (e.g. Borgeat as above, Section 6, 1st paragraph: since the users collaborating from different physical locations can be alone or in group, it is necessary to allow them to interact in a collaborative manner on the same virtual environment through the use of wall-sized displays; Examiner’s note: further, Figs.5-6 show the claimed virtual environment as well as use of a 3D avatar of a user).
In regards to medium claim 9 and system claim 16, claim(s) 9, 16 recite(s) limitations that is/are similar in scope to the limitations recited in claim 2. Therefore, claim(s) 9, 16 is/are subject to rejections under the same rationale as applied hereinabove for claim 2.
In regards to claim 3, the combination of Borgeat and Mattila teaches a method, wherein the virtual environment comprises an augmented reality (AR) environment comprising one or more AR holograms (e.g. Borgeat, Fig.6; Examiner’s note: Fig.6 shows display of real would picture of a user mixed in with the virtual environment; as such, may be viewed as also being an AR environment).
In regards to medium claim 10, claim(s) 10 recite(s) limitations that is/are similar in scope to the limitations recited in claim 3. Therefore, claim(s) 10 is/are subject to rejections under the same rationale as applied hereinabove for claim 3.
In regards to claim 4, the combination of Borgeat and Mattila teaches a method, wherein the first set of users are accessing the video conference session using 3D equipment (e.g. Borgeat, Section 6, 3rd paragraph: laser pointing interface is illustrated in Fig.5; it is composed essentially of a high-speed monochrome camera linked to a computer that detects the laser spot on the screen and computes the corresponding screen pixel; Examiner’s note: set of users access the video conference session using the 3D laser pointer and projection system that has a 3D stereoscopic display).
In regards to claim 6, the combination of Borgeat and Mattila teaches a method, further comprising:
storing 3D data of the 3D object in the video conference session (e.g. Borgeat, Section 3, 4th paragraph: the first step of the preprocessing is to decimate the model into a series of discrete LODs using an algorithm based on vertex pair contraction; the low resolution LOD is then decomposed into a set of triangle groups; each level is then recursively partitioned along the same borders as its lower resolution counterpart, and each group of the new level is subpartitioned to achieve the desired granularity; at the end of the process, we obtain a hierarchy of group subdivision spanning the whole sequence of LODs; groups are shaped based on criteria such as compactness, common orientation, texture/viewpoint association, and desired granularity (number of primitives per group); groups are individually converted into vertex-ordered triangle strips in order to maximize rendering speed); and
uncompressing the 3D object from the 2D data representation in the video conference session to the 3D data representation in the virtual environment based on the 3D data (e.g. Borgeat as above, Section 3, 4th paragraph: the low resolution LOD is then decomposed into a set of triangle groups; Examiner’s note: for a given resolution/LOD to be displayed, the data is uncompressed when the triangle sets are decomposed to be used for rendering (e.g. converted into vertex ordered triangle stripes)).
In regards to medium claim 13, claim(s) 13 recite(s) limitations that is/are similar in scope to the limitations recited in claim 6. Therefore, claim(s) 13 is/are subject to rejections under the same rationale as applied hereinabove for claim 6.
In regards to claim 7, the combination of Borgeat and Mattila teaches a method, further comprising:
providing one or more user interface elements for receiving an annotation from a video conference participant (e.g. Borgeat as above, Section 7, 1st paragraph: Fig.6 shows a video connection of a remote user, as described in Section 4, as well as the model annotation capability through the use of a 3D drawing system that writes directly on the 3D surface located under the cursor, controlled by the mouse or the laser pointer);
converting the annotation into an AR hologram (e.g. Borgeat, Fig.6/CAPTION: snapshot of a DIMENSION session, showing a remote user with his avatar and video window, as well as scene annotations drawn by the local and remote users; Examiner’s note: where the displayed annotations correspond to the claimed “AR hologram”); and
transmitting the AR hologram to the virtual environment for display (e.g. Borgeat, Fig.6/CAPTION: snapshot of a DIMENSION session, showing a remote user with his avatar and video window, as well as scene annotations drawn by the local and remote users; Examiner’s note: annotations would have to be transmitted to each participant in order for all remotely located users to consistently see the same annotations across all different displays).
In regards to medium claim 14 and system claim 18, claim(s) 14, 18 recite(s) limitations that is/are similar in scope to the limitations recited in claim 7. Therefore, claim(s) 14, 18 is/are subject to rejections under the same rationale as applied hereinabove for claim 7.
In regards to claim 11, the combination of Borgeat and Mattila teaches a medium, wherein the 3D object is editable in the virtual environment (e.g. Borgeat as above, Section 1, 2nd paragraph: infrastructure supports modification of the 3D model: a tool for annotating models during multi-user sessions is shown as an example; Section 7, 1st paragraph: Fig.6 shows a video connection of a remote user, as described in Section 4, as well as the model annotation capability through the use of a 3D drawing system that writes directly on the 3D surface located under the cursor, controlled by the mouse or the laser pointer).
Claim(s) 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Borgeat and Mattila as applied to claim 15 above, and further in view of Zingade et al. (US 2022/0237735 A1).
In regards to claim 19, the combination of Borgeat and Mattila teaches the system of claim 15, but does not explicitly teach the system, wherein the video conference session and virtual environment communicate via an SDK.
However, Zingade teaches a system, wherein the video conference session and virtual environment communicate via an SDK (e.g. [0002]: typical video conferencing applications, each participant is presented with a display that combines video streams (where available) from participants in the same video conference in the same graphical user interface, often in a montage or picture in picture arrangement; [0521]: deployment system 3606 may include software 3618, services 3620, hardware 3622, and/or other components, features, and functionality; in at least one embodiment, deployment system 3606 may include a software “stack,” such that software 3618 may be built on top of services 3620 and may use services 3620 to perform some or all of processing tasks, and services 3620 and software 3618 may be built on top of hardware 3622 and use hardware 3622 to execute processing, storage, and/or other compute tasks of deployment system 3606; [0550]: transfer of requests between services 3620 and inference applications may be hidden behind a software development kit (SDK), and robust transport may be provide through a queue).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Borgeat and Mattila to use an SDK, in the same conventional manner as taught by Zingade as both deal with video conferencing. The motivation to combine the two would be that it would allow the use of an SDK for requests and processing, providing the framework for the virtual video conferencing system.
In regards to claim 20, the combination of Borgeat and Mattila teaches the system of claim 15, but does not explicitly teach the system, wherein the 2D data representation is generated through an API.
However, Zingade teaches a system, wherein the 2D data representation is generated through an API (e.g. [0554]: provide a graphics rendering API and platform (e.g., for ray-tracing, 2D graphics, 3D graphics, and/or other rendering techniques to produce higher quality cinematics), and/or may provide other functionality for system 3700).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Borgeat and Mattila to use an API, in the same conventional manner as taught by Zingade as both deal with virtual video conferencing. The motivation to combine the two would be that it would allow the use of an API, to provide the functions for rendering purposes.
Allowable Subject Matter
Claim(s) 5, 12, 17 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim(s) 5, 12, 17 was/were carefully reviewed and a search with regards to independent claim(s) 1, 8, 15 has been made. Accordingly, those claim(s) are believed to be distinct from the prior art searched.
Regarding claim(s) 5, 12, 17 (and specifically independent claim(s) 1, 8, 15), the prior art search was found to neither anticipate nor suggest the method of claim 1/medium of claim 8/system of claim 15, further comprising: storing the 2D data representation in a virtual whiteboard in the video conference session; providing the virtual whiteboard for display in the video conference session; and enabling editing of the 2D data representation on the virtual whiteboard by one or more users in the virtual environment (emphasis added).
It is viewed that any of the previously cited references or any of the prior art searched, in part or in whole, cannot be combined in such a way to render the claimed invention obvious.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JED-JUSTIN IMPERIAL whose telephone number is (571)270-5807. The examiner can normally be reached Monday to Friday, 9am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JED-JUSTIN IMPERIAL/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
Prosecution Insights