DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The office action is in response to Applicant’s amendment filed 07/16/2026 which has been entered and made of record. Claims 1, 4-5, 9, 12-13, 17 and 20-21 have been amended. No claim has been newly added. Claims 1-24 are pending in the application.
Response to Arguments
Applicant’s arguments, filed 07/16/2026, on Page 13, last paragraph, with respect to the rejection(s) under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Yee and Kunert as fully explained below.
Applicant argues Yee, Bell, Kim and Holzer, taken individually or in combination, do not teach the newly amended independent claims.
Examiner agrees. However, a new ground of rejection is made in view of Yee and Kunert.
Conclusions: The rejections set in the previous Office Action are shown to have been proper, and the claims are rejected below. New citations and parenthetical remarks can be considered new grounds of rejection and such new grounds of rejection are necessitated by the Applicant's amendments to the claims. Therefore, the present Office Action is made final.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-6, 9, 11-14, 17 and 19-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yee et al. (US 20190083885 A1), hereinafter as Yee, in view of NPL Kunert et al. (“Multi-Window 3D Interaction for Collaborative Virtual Reality”), hereinafter as Kunert.
Regarding claim 1, Yee teaches A method comprising (paragraph [0011] “there is provided a method of controlling a virtual camera”): at an electronic device in communication with a display (paragraph [0077] “The electronic device 801 includes a display controller 807, which is connected to a video display 814, such as a liquid crystal display (LCD) panel or the like.”): concurrently displaying, using the display (Yee teaches displaying virtual camera locations and the scene together, paragraph [0118-0119] “Establishing constraints which may be used to configure the virtual camera in step 220, will now be described with reference to FIGS. 4A, 4B and 4C. FIG. 4A shows an example of a path 460 traced between two end points 440 and 450, using a touch gesture, in another example arrangement of the device 801. The path 460 is used to determine locations in the scene 410”): a three-dimensional model of a virtual three-dimensional environment; …… a representation of a viewpoint in the virtual three-dimensional environment and content including an image of the virtual three-dimensional environment from the viewpoint (Yee teaches the virtual modelled 3D scene and the virtual camera as the representation of a viewpoint, further teaches generating an image based on the virtual viewpoint and virtual 3D scene, paragraph [0069] ““Virtual cameras” are referred to as virtual because the functionality of the virtual cameras is computationally derived by methods such as interpolation between cameras or by rendering from a virtual modelled three dimensional (3D) scene constructed using data from many cameras (such as the cameras 120A to 120X) surrounding the scene (such as the arena 110), rather than simply the output of any single physical camera.”), wherein in accordance with a determination that the viewpoint in the virtual three-dimensional environment is a first viewpoint, the image is a first image from the first viewpoint (paragraph [0112] “In step 250, in response to detecting user input, the video display 814 is updated through synthesis of an updated virtual camera image. The updating of the display occurs due to the user input being detected, causing changes to one or more attributes or properties of the virtual camera…… The synthesis of virtual camera imagery uses a three-dimensional (3D) model of the scene generated from multiple physical cameras and the position of the virtual camera (usually three spatial dimensions) in scene space as well as the orientation of the virtual camera (usually another three rotational dimensions), and zoom.”); and in accordance with a determination that the viewpoint in the virtual three-dimensional environment is a second viewpoint different from the first viewpoint, the image is a second image from the second viewpoint (Yee teaches detecting user input in virtual camera control as the second viewpoint, further teaches updating the new image based on the second viewpoint input, paragraph [0112] “if the user touches a virtual camera control that has been configured in step 240, and this causes the virtual camera to move in space, synthesis of a new image would occur, showing to the user the newly updated point of view of the virtual camera.”).
Yee is not relied on for the below claim language …… in the three-dimensional model of the virtual three-dimensional environment, …… Kunert teaches …… in the three-dimensional model of the virtual three-dimensional environment, …… (Kunert teaches displaying 3D virtual avatar indicating different viewpoints in the virtual environment, further teaches the virtual portals based on the different viewpoints in the environment, Page 3272, Left Column, Figure 1, “The physical displays (wall and tabletop) and additional virtual views (portals) facilitate the simultaneous exploration of the virtual scene from different viewpoints. …… 3D video avatars represent the users in the virtual environment (see tabletop view).”).
Yee and Kunert are in the same field of endeavor, namely computer graphics, especially in the field of 3D virtual environment navigation. Kunert teaches a method of combining 3D virtual model and multiple virtual portals to improve user interaction and accuracy (Page 3271, Abstract “Our setup was tested in a formal user study, which revealed general advantages of collaborative 3D data exploration with multiple views in terms of user preference, comfort, and task performance.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Kunert with the method of Yee to improve user interaction and accuracy.
Regarding claim 3, Yee in view of Kunert teaches The method of claim 1, and further teach wherein the content includes video content that includes movement of the viewpoint in the virtual three-dimensional environment (paragraph [0130] “By marking the time with which points on the path 560 correspond, the speed at which movement of the virtual camera along the path 560 will occur may be inferred. Furthermore, by allowing the time markers 570 and 580 to be moved, it is possible to vary the speed at which the virtual camera will move along the path 560.”).
Regarding claim 4, Yee in view of Kunert teaches The method of claim 3, and further teach further comprising: capturing the video content (paragraph [0136] “A time slider 590 on the timeline 520 is used to move along the previously generated path 560 by increasing or decreasing the scene time. For example, the virtual camera may be configured to automatically follow an object at a certain distance, and through manipulation of the time slider 590 the user can preview the resultant video.”), including: while displaying the video content: receiving, via one or more input devices in communication with the electronic device, one or more inputs updating the viewpoint in the three-dimensional environment (Figure 5B and paragraph [0134] “the path 560 is input by adding two or more points using touch gestures. For each point on the path 560, a corresponding time marker may be added to the timeline control 510, and then both the intermediate spatial components and the intermediate temporal components of the locations of the path 560 may be inferred through interpolation methods.”); and in response to receiving the one or more inputs, updating the viewpoint in the three-dimensional environment in accordance with the one or more inputs and updating the video content in accordance with the viewpoint (Yee teaches updating virtual camera based on the user defined path and timeline to generate a video, paragraph [0136] “A time slider 590 on the timeline 520 is used to move along the previously generated path 560 by increasing or decreasing the scene time. For example, the virtual camera may be configured to automatically follow an object at a certain distance, and through manipulation of the time slider 590 the user can preview the resultant video.”).
Regarding claim 5, Yee in view of Kunert teaches The method of claim 3, and further teach further comprising: prior to capturing the video content, receiving, via one or more input devices in communication with the electronic device, one or more inputs defining a sequence of movement of the viewpoint in the virtual three-dimensional environment (paragraph [0119] “FIG. 4A shows an example of a path 460 traced between two end points 440 and 450, using a touch gesture, in another example arrangement of the device 801. The path 460 is used to determine locations in the scene 410. In one arrangement, only the start and end points are specified by the user, and the path is inferred by the device 801.”); and after receiving the one or more inputs, capturing the video content, including updating the viewpoint in the three-dimensional environment in accordance with the one or more inputs and updating the video content in accordance with the viewpoint (Yee teaches updating virtual camera based on the user defined path and timeline to generate a video, paragraph [0129-0130] “The association of temporal information with the locations determined through user interaction in step 220, will now be described with reference to FIGS. 5A, 5B and 5C. FIG. 5A shows a user interaction creating a path 560 which is also associated with a timeline control 510”, paragraph [0136] “A time slider 590 on the timeline 520 is used to move along the previously generated path 560 by increasing or decreasing the scene time. For example, the virtual camera may be configured to automatically follow an object at a certain distance, and through manipulation of the time slider 590 the user can preview the resultant video.”).
Regarding claim 6, Yee in view of Kunert teaches The method of claim 3, and further teach further comprising: displaying, using the display, a plurality of control elements associated with the viewpoint in the virtual three-dimensional environment (Yee teaches a slider in Fig. 5B as one control element for virtual camera path, paragraph [0056] “FIG. 5B shows a time slider displayed on the electronic device”, paragraph [0136] “A time slider 590 on the timeline 520 is used to move along the previously generated path 560 by increasing or decreasing the scene time.”); and receiving, via one or more input devices in communication with the electronic device, one or more inputs directed to the plurality of control elements (paragraph [0078] “The electronic device 801 also includes user input devices 813 which are typically formed by keys, a keypad or like controls. In one arrangement, the user input devices 813 include a touch sensor 829 physically associated with the video display 814 to collectively form a touch-screen.”, paragraph [0137] “Thus, the control of the time slider 590 by the user during step 250 suffices to control the spatial location of the virtual camera in the scene, merely by sliding a thumb along the timeline 520.”), wherein the movement of the viewpoint in the three-dimensional environment in the video content is based on the one or more inputs directed to the plurality of control elements (Yee teaches updating virtual camera based on the user defined path and timeline to generate a video, paragraph [0129-0130] “The association of temporal information with the locations determined through user interaction in step 220, will now be described with reference to FIGS. 5A, 5B and 5C. FIG. 5A shows a user interaction creating a path 560 which is also associated with a timeline control 510”, paragraph [0136] “A time slider 590 on the timeline 520 is used to move along the previously generated path 560 by increasing or decreasing the scene time. For example, the virtual camera may be configured to automatically follow an object at a certain distance, and through manipulation of the time slider 590 the user can preview the resultant video.”).
Regarding claim 9, it recites similar limitations of claim 1 but in an electronic device form. The rationale of claim 1 rejection is applied to reject claim 9. In addition, Yee teaches An electronic device comprising: memory; and one or more processors coupled to the memory and configured to perform a method comprising (paragraph [0061] “FIGS. 8A and 8B collectively form a schematic block diagram representation of an electronic device upon which described arrangements can be practised.”, paragraph [0030-0033] “According to still another aspect of the present disclosure, there is provided a tablet device adapted to configure a virtual camera, comprising: a touchscreen; a memory; a processor configured to execute code stored on the memory”):
Regarding claim 11, claim 11 has similar limitations as claim 3, therefore it is rejected under the same rationale as claim 3.
Regarding claim 12, claim 12 has similar limitations as claim 4, therefore it is rejected under the same rationale as claim 4.
Regarding claim 13, claim 13 has similar limitations as claim 5, therefore it is rejected under the same rationale as claim 5.
Regarding claim 14, claim 14 has similar limitations as claim 6, therefore it is rejected under the same rationale as claim 6.
Regarding claim 17, it recites similar limitations of claim 1 but in a non-transitory computer readable storage medium form. The rationale of claim 1 rejection is applied to reject claim 17. In addition, Yee teaches A non-transitory computer readable storage medium storing instructions that, when executed by an electronic device including memory and one or more processors coupled to the memory causes the electronic device to perform a method comprising (Figure 8A and 8B, paragraph [0016] “there is provided a non-transitory computer-readable medium having a computer program stored thereon for configuring a virtual camera”):
Regarding claim 19, claim 19 has similar limitations as claim 3, therefore it is rejected under the same rationale as claim 3.
Regarding claim 20, claim 20 has similar limitations as claim 4, therefore it is rejected under the same rationale as claim 4.
Regarding claim 21, claim 21 has similar limitations as claim 5, therefore it is rejected under the same rationale as claim 5.
Regarding claim 22, claim 22 has similar limitations as claim 6, therefore it is rejected under the same rationale as claim 6.
Claim(s) 2, 10 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yee et al. (US 20190083885 A1), hereinafter as Yee, in view of NPL Kunert et al. (“Multi-Window 3D Interaction for Collaborative Virtual Reality”), hereinafter as Kunert, further in view of Bell et al. (US 20230306688 A1), hereinafter as Bell.
Regarding claim 2, Yee in view of Kunert teaches The method of claim 1, however, Yee and Kunert are not relied on for the below claim language further comprising: in accordance with a determination that the image is displayed as a two-dimensional image, displaying, using the display, a selectable option that, when selected, causes the electronic device to display the image as a three-dimensional image; and in accordance with a determination that the image is displayed as a three-dimensional image, displaying, using the display, a selectable option that, when selected, causes the electronic device to display the image as the two-dimensional image.
Bell teaches further comprising: in accordance with a determination that the image is displayed as a two-dimensional image, displaying, using the display, a selectable option that, when selected, causes the electronic device to display the image as a three-dimensional image (Bell teaches selectable button, and transition between 3D imagery data and 2D images, paragraph [0120] “A triggering event can comprise user input, navigation to a field of view or portion of a model, mousing over or hovering over an object, selecting an object (e.g., selecting a button, selecting an image, selecting a portion of a model, etc.), determining that a 2D image is available for view, and the like.”, paragraph [0147] “At 808, a system can, in response to a trigger, transition (e.g., via transitioning component 636) between 3D imagery data and 2D imagery data. In embodiments, a system can detect a trigger based on occurrence of a triggering event. For example, the triggering event can comprise data representing user input (e.g., from a touch screen device, a mouse, a keyboard, a button, a microphone, a camera, or other interface device)”, paragraph [0154] “Transitioning between rendering the 3D model and rendering the 2D image can comprise rendering the 2D image in place of the 3D model, rendering a combination of a 3D model and 2D images, and the like.”); and in accordance with a determination that the image is displayed as a three-dimensional image, displaying, using the display, a selectable option that, when selected, causes the electronic device to display the image as the two-dimensional image (Bell paragraph [0037] “A selection component can facilitate selecting the 2D imagery data for display. For example, the selection component can select a 2D image for display based on selection criteria. In another aspect, the selection component can select, based on 3D imagery data and associated viewpoints of a 3D model (e.g., such as a viewpoint of the 3D model or a selected position of the 3D model), 2D imagery for display in a 2D mode.”).
Yee, Kunert and Bell are in the same field of endeavor, namely computer graphics, especially in the field of 3D virtual environment navigation. Bell teaches a method of transition between 3D model display and 2D image display to improve user satisfaction (Bell paragraph [0043] “Such embodiments can enable additional features in 3D modeling systems, improve user satisfaction, and provide other benefits that will be apparent herein.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Bell with the method of Yee and Kunert to improve user satisfaction.
Regarding claim 10, claim 10 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2.
Regarding claim 18, claim 18 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2.
Claim(s) 7, 15 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yee et al. (US 20190083885 A1), hereinafter as Yee, in view of NPL Kunert et al. (“Multi-Window 3D Interaction for Collaborative Virtual Reality”), hereinafter as Kunert, further in view of Kim et al. (US 20230162434 A1), hereinafter as Kim.
Regarding claim 7, Yee in view of Kunert teaches The method of claim 3, Yee and Kunert are not relied on for the below claim language wherein the movement of the viewpoint in the virtual three-dimensional environment in the video content is based on movement of a physical camera that captured real video footage.
Kim teaches wherein the movement of the viewpoint in the virtual three-dimensional environment in the video content is based on movement of a physical camera that captured real video footage (Kim paragraph [0119] “As described above, the virtual viewpoint trajectory generated based on the real viewpoint trajectory can be generated according to the realistic movement of the real camera, and thus it is not necessary to manually design the virtual viewpoint trajectory.”).
Yee, Kunert and Kim are in the same field of endeavor, namely computer graphics, especially in the field of 3D augmented and virtual environment. Kim teaches a method of generating virtual viewpoint trajectory based on real camera trajectory to improve accuracy (paragraph [0028] “the camera motion estimation method and system for an augmented reality tracking algorithm according to an embodiment of the present invention can evaluate the accuracy of an object tracking algorithm using a generated image sequence and improve tracking performance by modifying parameters.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Kim with the method of Yee and Kunert to improve accuracy.
Regarding claim 15, claim 15 has similar limitations as claim 7, therefore it is rejected under the same rationale as claim 7.
Regarding claim 23, claim 23 has similar limitations as claim 7, therefore it is rejected under the same rationale as claim 7.
Claim(s) 8, 16 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yee et al. (US 20190083885 A1), hereinafter as Yee, in view of NPL Kunert et al. (“Multi-Window 3D Interaction for Collaborative Virtual Reality”), hereinafter as Kunert, further in view of Kim et al. (US 20230162434 A1), hereinafter as Kim, and Holzer et al. (US 20210227195 A1), hereinafter as Holzer.
Regarding claim 8, Yee in view of Kunert and Kim teaches The method of claim 7, however, Yee in view of Kunert and Kim is not replied on for the below claim language, further comprising: presenting, using the display, a second video content that concurrently includes a portion of the video content of the virtual three-dimensional environment and a portion of the real video footage. Holzer teaches further comprising: presenting, using the display, a second video content that concurrently includes a portion of the video content of the virtual three-dimensional environment and a portion of the real video footage (Holzer teaches generating a view based on a virtual viewpoint, further teaches combining the generated view with real video footage, Yee teaches generating a virtual video based on sequence of virtual camera, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Holzer with Yee and Kim. Holzer paragraph [0074] “In some embodiments, a view of an object from a novel viewpoint may be generated. For instance, the multi-view capture may include a number of static images of the object each captured from a respective perspective viewpoint. Paragraph [0225] “native video data from a live video can be augmented with virtual data to create synthetic images and then output in real-time.”).
Yee, Kunert, Kim and Holzer are in the same field of endeavor, namely computer graphics, especially in the field of 3D virtual environment navigation. Holzer teaches a method of combining real video footage with newly generated virtual image to improve user interaction and accuracy (paragraph [0125] “the ability to use the distinction between content and context for improvements in the user-experience; the ability to use the distinction between content and context for improvements in memory footprint (an example would be high quality compression of content and low quality compression of context); the ability to associate special feature descriptors with MVIDMRs that allow the MVIDMRs to be indexed with a high degree of efficiency and accuracy; and the ability of the user to interact and change the viewpoint of the MVIDMR.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Holzer with the method of Yee, Kunert and Kim to improve user interaction and accuracy.
Regarding claim 16, claim 16 has similar limitations as claim 8, therefore it is rejected under the same rationale as claim 8.
Regarding claim 24, claim 24 has similar limitations as claim 8, therefore it is rejected under the same rationale as claim 8.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMING WEI whose telephone number is (571)272-3831. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/XIAOMING WEI/ Examiner, Art Unit 2611
Prosecution Insights