Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant’s arguments with respect to claim(s) 1 and 16 have been considered but are moot because they relate to newly amended claim limitations for which new art Tardif is provided.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim Rejections - 35 USC § 103
Claim(s) 1-6 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al (2022/0379197) in view of Tardif et al (2011/0267269).
In regard to claim 1 Zhang discloses an augmented reality apparatus (Zhang Fig. 2 note Gamepad 2 as the apparatus which is part of the VR system), comprising:
a memory (Zhang Fig. 1 and par. 74 note storage unit 223); and
a processor coupled to the memory (Zhang Fig 1. And par. 74 note second processor 23) , wherein the processor is configured to:
obtain a first image from a first camera, the first image being associated with a first capture time based on a first clock (Zhang Fig. 7 and pars 91-93 note time stamp added to second cameras of gamepad);
obtain a second image from a second camera of another device, the second image including a third capture time based on the second clock (Zhang Fig. 7 and pars 91-93 note time stamp added to the first cameras of the HMD).
Zhang further discloses determining time deviations between camera images. It is noted that Zhang does not disclose details of a common clock. However, Tardif discloses a method of image sensor synchronization (Tardif Fig. 5 and generally pars 77-82) including
obtaining a first image from a first camera associated with a first clock (Tardif par. 78 note time stamped image captured by sensor A)
map the first capture time to a second clock to obtain a second capture time, wherein the second clock is a common clock corresponding to a common network time and wherein the second capture time is based on the common network time (Tardif par. 78 note marking images received from sensor A with a time stamp generated by a local clock of the computing device)
obtain a second image from a second camera associated with a third capture time (Tardif par. 78 note time stamped image captured by sensor B)
determine phase delta information based on a time difference between a second capture time associated with the first image and a third capture time of the second image wherein the phase delta information aligns the second capture time and the third capture time with the common network time (Tardif par. 78 note determining whether sensors A and B are out of sync with each other, or are out of sync with the local clock); and
output the phase delta information to adjust the next capture time of at least one of the first camera and the second camera (Tardif par. 78 note sending a command to the sensors to adjust a vertical blanking interval in order to adjust the next capture time based on the determined phase, or synchronization, difference)
It is therefore considered obvious that one of ordinary skill in the art would recognize the advantage of incorporating camera synchronization control as taught by Tardif in the invention of Zhang in order to allow for control over both synchronization between cameras and between the cameras and the common clock as suggested by Tardif (Tardif par. 78).
In regard to claim 2 refer to the statements made in the rejection of claim 1 above. Zhang further discloses the augmented reality apparatus further comprises the first camera (Zhang Fig. 1 and par 57 note second camera 21).
In regard to claim 3 refer to the statements made in the rejection of claim 1 above. Tardif further discloses that:
the phase delta information includes information to adjust the next capture time of the first camera (Tardif pars 78 and 80 note steps 525-535 determining whether an interval between sensor A and sensor B or between the sensors and the host requires adjustment and step 540 calculating the adjustment); and
the processor is configured to adjust a vertical blanking period of the first camera based on the phase delta information (Tardif pars 78 and 80 note step 550 note adjusting a vertical blanking interval of the sensor according to the calculated adjustment)
In regard to claim 4 refer to the statements made in the rejection of claim 1 above. Tardif further discloses:
determining the network time (Tardif par. 78 note local clock); and
broadcasting the network time (Tardif fig. 2 and par. 41 note clock provided by computing environment to the capture device 20).
In regard to claim 5 refer to the statements made in the rejection of claim 1 above. Tardif further discloses that:
the phase delta information includes information to adjust the next capture time of the second camera (Tardif pars 78 and 80 note steps 525-535 determining whether an interval between sensor A and sensor B or between the sensors and the host requires adjustment and step 540 calculating the adjustment); and
the processor is configured to transmit the phase delta information to the device to adjust a adjust a vertical blanking period of the second camera (Tardif pars 78 and 80 note step 550 note adjusting a vertical blanking interval of the sensor according to the calculated adjustment, further note par. 78 note computing device may send a command to the sensor to adjust a vertical blanking interval)
In regard to claim 6 refer to the statements made in the rejection of claim 1 above. Tardif further discloses that the common network time comprises a timing synchronization function (TSF) time (Tardif pars 78-80 note the transmitted adjustments are used to synchronize the camera images and thus comprises a ‘timing synchronization function’).
Claims 16 and 18 relate to a method corresponding to the process steps implemented by the memory and processor in the apparatus of claims 1 and 4 above. Refer to the statements made in regard to claims 1 and 4 above for the rejection of claims 16 and 18 which will not be repeated here for brevity.
Claim(s) 7-11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Tardif and Chen (2019/0035149).
In regard to claims 7 and 20 refer to the statements made in the rejection of claims 1 and 16 above. Zhang does not disclose details regarding modeling. However, Chen discloses generating personalized 3D models of a user including:
obtain a third image, the third image including at least a head and torso of a person (Chen Fig. 1 note full body images input into ‘Personalized 3D Body Modeling’ module, further note Figs 27-28 and pars 353-360 for a first full body method using fully body images from e.g. a Kinect camera, finally note pars 394-397 for a second full body method using full body images captured by a mobile device, further note full body images include the head and torso);
extract a first set of features form the third image (Chen par. 357 note the first full body method determining 30 morph and 18 joint parameters for depth based images, also note par. 395 the second full body method extracting image features for mobile images);
obtain a fourth image, the fourth image including a portion of a head around eyes of the person (Chen generally Figs 1-26 and pars 196-344 for multiple methods of obtaining fourth images of a user’s face including the eye portion of the face including pars 202-229 for a first head modeling method using a single image and pars 230-319 for a second head modeling method using multiple images, particularly note pars 320-329 program to assist in obtaining one or multiple images of a user’s face with acceptable quality);
extract a second set of features from the fourth image (Chen pars. 209 and 240 note extracting facial landmarks);
generate a head model based on the first set of features and second set of features, the head model including pose information for a camera that captured the third image (Chen pars. 230-259 note second head modeling method using a second set of features, or landmarks, detected in the face images, further note par. 268 for using camera pose information in generating a head model, finally note par. 314 for using a first set of features from the full body images to determine the size of the head model);
generate a body model based on the third image (Chen pars 353-397 for generating full body models);
combine the head model and body model into a full body model (Chen pars 406-414 note combining the head and body models); and
output the full body model (Chen par. 192 note outputting a single unified 3D user avatar).
It is therefore considered obvious that one of ordinary skill in the art before the effective filing date of the invention would recognize the advantage of incorporating the 3D model generation of Chen into the invention of Zhang and Tardif in order to allow a user to generate a 3D body avatar as suggested by Chen (Chen pars 100-101)
In regard to claim 8 refer to the statements made in the rejection of claim 7 above. Chen further discloses that the processor is further configured to concatenate the first set of features and the second set of features (Chen pars 406-414 note combining the head and body model also concatenates the features of those models) .
In regard to claim 9 refer to the statements made in the rejection of claim 8 above. Chen further discloses the processor is further configured to generate the head model based on the concatenated first set of features and second set of features (Chen par. 314 note features of the body model are used in generating the head model).
In regard to claim 10 refer to the statements made in the rejection of claim 7 above. Chen further discloses that the processor is further configured to normalize a color of the third image (Chen par. 403 note normalizing skin tone colors between the full body and face images).
In regard to claim 11 refer to the statements made in the rejection of claim 7 above. Chen further discloses that the head model further includes a mesh model of the head and corresponding texture (Chen pars 260-264 note head model includes a mesh model and textures).
Allowable Subject Matter
Claim 12-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 12 requires, in addition to the limitations of the claims from which it depends, segmenting third and fourth images to identify portions of the images corresponding to a HMD worn by the user.
The closest arts are Zhang and Chen. Zhang discloses a virtual reality system including an HMD and controllers with cameras and includes details of synchronizing the cameras. Chen discloses a method for generating a personalized 3D avatar including taking third and fourth images to generate head and body models of a user. However neither Zhang nor Chen disclose details of identifying and segmenting an HMD in the third and fourth images of the user as required by claim 12.
Claims 13-15 depend from claim 12 and include allowable subject matter for the same reasons.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMIAH CHARLES HALLENBECK-HUBER whose telephone number is (571)272-5248. The examiner can normally be reached Monday to Friday from 9 A.M. to 5 P.M.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEREMIAH C HALLENBECK-HUBER/ Primary Examiner, Art Unit 2481
Prosecution Insights