DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1 and 22-29 of the instant application 18/916,427 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-11 of U.S. Patent 12,149,671. Although the claims at issue are not identical, they are not patentably distinct from each other as seen by a representative sample of comparative analysis between the instant claim 1 and the issued claim 1.
The instant application: 18/428,549
U.S. Patent 12,149,671
Claim 1. (currently amended) A method for a telepresence system, the method comprising:
capturing video from a plurality of cameras at a source location, the video including a subject within a respective point of view;
at each of a respective at least one remote location determining a viewpoint of a respective Virtual Reality (VR) or Augmented Reality (AR) headset worn by a remote user, the viewpoint corresponding to a determination of a head position and a head orientation of the remote user based on an orientation tracking system associated with the headset;
creating a synthetic three-dimensional video from the captured video, the synthetic three-dimensional video including the subject and corresponding to the determined viewpoint of the respective remote user;
at each of the respective at least one remote location, capturing a local scene corresponding to the determined viewpoint of the respective remote user by respective local cameras mounted on the respective headset:
composting the local scene with the synthetic three-dimensional video: and displaying the composted local scene with the synthetic three- dimensional video of the subject to the respective headset at the at least one remote location.
1. A one-way telepresence method, comprising: capturing three-dimensional video from a plurality of stationary cameras at a source location, the video including a subject within a respective point of view; at each of a respective at least one remote location, determining a viewpoint of a respective Virtual Reality or Augmented Reality headset worn by a remote user, the viewpoint corresponding to a determination of the head position and head orientation of the remote user based on an orientation tracking system associated with the headset; transmitting the viewpoint of the respective remote user from the at least one remote location to the source location; at the source location, creating at least one synthetic three-dimensional video from the captured three-dimensional video, the synthetic three-dimensional video including the subject and corresponding to the determined viewpoint of the respective remote user; the creating including: a) applying at least one of i) realtime view interpolation techniques to at least a portion of the captured video three-dimensional video and ii) a light field rendering technique to at least a portion of the captured three-dimensional video from the plurality of stationary cameras and; b) integrating at least one of i) chromakey and ii) depth filtering technologies; transmitting the synthetic three-dimensional video of the subject to the at least one remote location; and displaying the synthetic three-dimensional video of the subject to the respective headset at the at least one remote location; wherein the one-way telepresence system including a plurality of remote locations with respective headsets worn by different users; and the synthetic video for each respective headset is different to correspond with the determined viewpoint for each user.
From the above evidence, it is clear that the claim limitations of the instant application are covered by the limitations of claimed invention of the issued patent. It would have been obvious to the ordinary artisan before the effective filing date to broaden the claim in order to enjoy greater coverage and protection.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 and 22-29 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1, inter alia, recites “composting” and “composted”. The Specs does not disclose what they are. Examiner assumes a potential typo and probably applicant meant “compositing” and “composited” since the Specs does disclose the compositing and composited. Correction is required.
By definition of Composting vs Compositing
Composting and compositing are completely different processes — one is an environmental waste management and soil improvement method, while the other is a general term for combining or assembling components. For example, composting is the natural decomposition of organic matter such as food scraps, yard trimmings, and other biodegradable materials into a nutrient-rich soil amendment called compost while Compositing is a general term for combining or assembling different components into a single unit or structure. Based on this definition and the Specs, examiner will examine the application based on the best mode of analysis, i.e., compositing, emerging, combining and/or integrating.
Allowable Subject Matter
Claims 25-26 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 22-24 and 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Lalonde et al (US 2017/0287215) in view of Grau et al (US 2018/0158246).
Claim 1. Lalonde teaches a method for a telepresence system, the method comprising:
capturing video from a plurality of cameras at a source location, the video[[s]] including a subject within a respective point of view; (Lalonde: Image content captured by the pass-through cameras can be combined with virtual content in a display of the HMD device configured to provide a number of graphical user interface (GUI) configurations, [0020]. A tracking device (not shown), such as, for example, a camera, can be positioned in a physical, real world space, and can be oriented to capture as large a portion of the room 304 as possible with its field of view, [0070]);
at each of a respective at least one remote location determining a viewpoint of a respective Virtual Reality (VR) or Augmented Reality (AR) headset worn by a remote user, the viewpoint corresponding to a determination of a head position and a head orientation of the remote user based on an orientation tracking system associated with the headset; (Lalonde: [0028] A Virtual Reality (VR) system and/or an Augmented Reality (AR) system may include, for example, a head-mounted display (HMD) device, VR headset, or similar device or combination of devices worn by a user to generate an immersive virtual world environment to be experienced by the user, [0028]; Please note HMD 104 comprises at least three cameras, [0034-0035]);
creating a synthetic three-dimensional video from the captured video, the synthetic three-dimensional video including the subject and corresponding to the determined viewpoint of the respective remote user; (Lalonde: The systems and methods described herein can obtain and use knowledge regarding user context including information about a physical world (i.e., real world) surrounding the user, a location and/or gaze associated with the user, and context about virtual content being providing within the HMD device worn by the user. Such information can allow a VR director or VR content creator to adapt virtual content to allow for determined user context when providing real world (i.e., physical world) image and/or video content in pass-through content areas without detracting the user from the immersive virtual experience, [0031]. And since Lalonde teaches, “The virtual content for the display region 108 may be generated from images, video, computer graphics, or other media, [0032]. Examiner maps the virtual content to synthetic content and wishes to provide additional reference to support the mapping as Grau teaches, “While the term avatar may typically refer to an animation or cartoon-like representation of a person in a virtual or augmented reality versus a photo realistic representation, character, or model of a person, for simplicity and consistency sake, avatar as used herein may refer to either a synthetic avatar (SA) such as an animation or a photo-realistic avatar (PRA) that is generated by using video of a user, [0029]);
at each of the respective at least one remote location, capturing a local scene corresponding to the determined viewpoint of the respective remote user by respective local cameras mounted on the respective headset: (Lalonde: Fig. 2 shows HMD 204 comprises multiple cameras 202 wherein The cameras 202 can capture and provide images and virtual content over a network 206, or alternatively, can provide images and virtual content to an image processing system 208 for analysis, processing, and re-distribution to HMD device 204 over a network such as network 206. In some implementations, the cameras 202 can feed captured images directly back into the HMD device 204. For example, in the event that cameras 202 are configured to operate as pass-through cameras installed to capture still images or video of an environment surrounding a user wearing the HMD device 204, the content captured by the pass-through cameras can be directly transmitted and displayed on the HMD device 204 or processed by system 208 and displayed on the HMD device 204, [0048] where communication between users (at least first user and second users takes place, [0046]). Precisely, HMD 104 comprises at least three cameras, [0034-0035]. Similarly, Grau: Fig. 3 shows multiple cameras, 308R, 308L. Here, please see the objection above regarding the language of the claim, “composting”. Examiner believes that the applicant meant to be “compositing” rather. Examiner reads the “compositing” as merging as shown in Grau and as compositing in Lalonde “The compositing may be used to create the illusion that the content originates from portions of the same scene. In some implementations, the composited image content is a combination of a virtual scene and a physical world scene viewed by the user or (viewed by a pass-through camera). The composited content may be obtained or generated by the systems described herein. In some implementations, compositing image content includes the replacement of selected parts of an image with other content from additional images, [0051]).
composting the local scene with the synthetic three-dimensional video: While Lalonde presents compositing image content includes combining of visual content (e.g., virtual objects, video footage, captured images, and/or scenery) from separate sources into at least one view for display. The compositing may be used to create the illusion that the content originates from portions of the same scene. In some implementations, the composited image content is a combination of a virtual scene and a physical world scene viewed by the user or (viewed by a pass-through camera). The composited content may be obtained or generated by the systems described herein. In some implementations, compositing image content includes the replacement of selected parts of an image with other content from additional images. Similarly, GRAU teaches, “[0111] Now returning again to process 600 that may include “merge image data of occluded parts of face with rest of frame” 620. Now, the occluded parts then may be merged with the remainder of the face, if not done already, then head, and then body. Also if treated separately and not treated integrally within the exterior images, the resulting full body avatar then may be merged with a background scene image partly or wholly taken from the external images”, [0111]; The process 1200 may include “provide image for merging and display” 1220, and the final image with the occluded area may be merged with other views of the same perspective including the remainder of the face, head and body to form an entire avatar, and then merging with a background shown in the exterior images. Also as described above, the final refined image may be provided to form a view of the virtual or augmented reality that can be registered with other perspectives of the reality to form a 3D space for the reality. Then images of other perspectives of the 3D space can be formed for other HMDs networked to the HMD providing the internal images, [0126, 0181]
displaying the composted local scene with the synthetic three- dimensional video of the subject to the respective headset at the at least one remote location.
(Lalonde: hardware compositing layer operable to display image content retrieved from the plurality of pass-through cameras and composite the image content within virtual content displayed on the head-mounted display device, [0010]. Similarly, Grau: Process 600 may include “provide frame for display” 622, where the final image or frame is provided for further post-processing and then to a display controller to display on the HMDs of the users other than the user of the HMD with the internal cameras, [0113]).
Therefore, it would have been obvious to the ordinary artisan to incorporate the teaching of Grau into the teaching of Lalonde for the purpose of explicitly detailing the merging (i.e., compositing) process for greater enhancement of communication between users.
Claim 22. (new) The method of claim 1wherein the creating includes: a) applying at least one of i) real time view interpolation techniques to at least a portion of the captured video (Grau: applying interpolation algorithms, [0107, 0125]) and ii) a light field rendering technique to at least a portion of the captured video from the selected subset of the plurality of cameras; and b) integrating at least one of i) chromakey (Please also see the parent case, 17/737,841- The Final Office Action filed 3/14/24 detail the chromakey) and ii) depth filtering technologies (Grau: Track depth-sensing with the operation of filtering, [0038, 0063]).
Claim 23. (new) The method of claim 1wherein the telepresence system includes a plurality of remote locations with respective headsets worn by different users and the synthetic three-dimensional video for each respective headset is different to correspond with the determined viewpoint for each user. (Please also see the parent case, 17/737,841- The Final Office Action filed 3/14/24 details the similar feature in claim 29. Grau: the HMD may be used to perform a run-time or use stage versus the learning or training stage. Now the HMD is worn by at least one user, and most likely a number of users where the HMDs are networked together to view different perspectives of the same virtual or augmented reality, [0098]).
Claim 24. (new) The method of claim 1wherein the synthetic three-dimensional video includes a single synthetic video of the subject for the respective headset to display to both eyes of the respective remote user based on the determined viewpoint of the respective remote user being at least at a threshold distance from the subject. (Lalonde: the process 600 may include detecting a number of physical objects in particular image content in which the objects are within a threshold distance from the user operating the HMD device 204, for example, [0093]).
Claim 27. (new) The method of claim 1further comprising: selecting a subset of the plurality of cameras that most closely correspond to the determined viewpoint of the respective remote user; and creating the synthetic three-dimensional video from the captured video of the selected subset of the plurality of cameras. (Grau: Both the HMDs 404 and external cameras 402 are communicatively connected, either wirelessly or wired, to an image processing unit 410 that performs the method operations. The image processing unit 410 may be considered one or more separate devices. Thus, the image processing unit 410 may be a game box, TV box (e.g., a cable or satellite box), computer, remote server, smartphone, tablet, and so forth. Alternatively, the image processing unit 410 may be part of one or more of the HMDs or one or more of the cameras mentioned here such as the external cameras. In this case, the external cameras even may be mounted on the HMD itself to record at least the non-occluded parts of the face whether by an arm attaching the external camera(s) to the HMD or mounted directly on the HMD, [0037]. Here examiner maps the external cameras to the subset of cameras).
Claim 28. (new) The method of claim 1wherein the local scene is processed by a chromakey system to exclude an item of a key color from the processed local scene. (Please see claim 22 above or also see the parent case, 17/737,841- The Final Office Action filed 3/14/24 detail the chromakey).
Claim 29. (new) The method of claim 28 wherein the excluded item is replaced with a corresponding virtual item in the processed local scene. (Grau: Process 900 may include “warp occluded face parts from images to 3D model” 912. Specifically, the result is a mapping to effectively remove the pixel data that represents the HMD itself, and replace it with the image data from the internal images that show the occluded area of the user's face. The process includes a warping (or re-projection) of the captured examples as a normalization operation here. Thus, the face model here may be in a standard position, for example, looking straight forward, [0081]).
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUNG-HOANG J. NGUYEN whose telephone number is (571)270-1949. The examiner can normally be reached Reg. Sched. 6:00-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUNG-HOANG J NGUYEN/Primary Examiner, Art Unit 2691
Prosecution Insights