Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims pending are 1, 4, 6-8, 11, 13-15, 18, 20, 22-24 and 26-30 (Claims 2-3, 5, 9-10, 12, 16-17, 19, 21, 25 and 31-32 have been cancelled).
Response to Arguments
Applicant’s arguments with respect to claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The examiner previously cited BALLARD et al., US 20220217301, which was used to reject dependent claims (27-32, where 31-32 are now cancelled) which disclose a camera calibration system which may be preregistered or calibrated (preexisting) and update the calibration parameters and also use an autostereoscopic display (para 42)
US 20240104778 disclose (abstract) the features of camera calibration which updates the calibration parameters for recalibration.
US 20140104368 (para 25) disclose a telepresence system which includes multiple displays for providing multiple perspective and viewpoints to enhance meeting interaction.
US 2030182083 discloses a system where the display can be calibrated using a reflection of the display via a mirror to the attached cameras on the display. (Fig 4, para 69).
Claim Rejections - 35 USC § 112
Claims 1, 4, 6-8, 11, 13-15. 18, 20, 22-24, and 26-30 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The examiner notes the claims have been amended to include “preexisting” calibration. A review of the specification/drawings, the examiner is unable to find support for this language.
The examiner notes it is conventional for cameras to have a factory setting (set of calibration parameters) as well as cameras/system that update parameters of the cameras. Thus it is unclear from the amendment/disclosure what the “preexisting” is actually referring to without a description/support.
For rejection purposes the preexisting is being rejected as cameras which contain calibration data (either factory settings or updating parameters as done when cameras get out of calibration (re-calibrated)).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6-8, 13-15, 20, 22, 24, 26-27, 29 and 33 is/are rejected under 35 U.S.C. 103 as being unpatentable over CLAVEU et al., US 20170287166 and TAO et al., US 20220164987 in view of BALLARD et al., US 20220217301.
PNG
media_image1.png
453
660
media_image1.png
Greyscale
The claimed receiving…is met by cameras 44a/b/c/d (Fig 17) where the cameras have an overlapped field (para 32, 64, 77, 95, 125-129 and capture a scene of a calibration target 20, including various poses (orientation) of target 20 (para 158). As noted by CLAVEU the coupling may include wireless or wired connections, where the cameras are coupled to each other via processing unit 80 (para 158)
The claimed determining a position of the target…including capturing images of the target from the respective cameras and using information of the target for positional information of the target (para 48, 76, 89, 103, 110, 113, 118, 124, 133, 137, 146, 151-154. Where the alignment is based upon the partial overlapped fields of view and the extrinsic (external position) of cameras being adjusted (para 32, 106, 124, 125, 128, 132, 138, 140-146, 158).
c/d) measuring a first and second reprojection error…is met where the reprojection error for intrinsics and extrinsic camera parameters are calculated based upon the target (checkerboard/fudicial) (para 70, 121, 122, 142-146, 149, 154) and calculating the position and orientation of each camera with respect to on another (para 124).
e) providing an indication…a quality indicator (para 31, 131, 143-146, 151-154)
Regarding e) the examiner notes CLAVEU discloses threshold and quality indicator, however does not explicitly recite great than a threshold, although well-known, the examiner provides evidence of such via Tao.
The examiner incorporates TAO et al., US 20220164987 which explicitly recites (para 91) if an error is larger than a threshold providing a calibration result indicating unsuccessful or successful calibration, which if unsuccessful is indicated that provides indication that calibration (a recalibration/new calibration) should be performed.
Regarding the newly added: new calibration should be performed as noted about both CLAVEU and TAO provide indication of quality, thus if quality indicates whether successful or not, wherein unsuccessful is an indication that calibration should be performed (recalibration).
The motivation to modify CLAVEU with TAO provides the conventional ability to notify the user(s) whether calibration has been met (Pass) or not (FAIL) in order to ensure the user(s) is fully aware/informed of the calibration results being performed, and recalibrate as necessary to ensure a proper operating parameters of the system, thereby notifying the user(s) of the status of such calibration which indicates if calibration should be performed (unsuccessful indicator), thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding the newly amended limitations:b)
Although the above combination CLAVEU/TAO does not disclose such display, where TAO discloses using a default extrinsic parameter when calibration is unsuccessful (para 95), default being a preset/factory/stored parameter (preexisting).
PNG
media_image2.png
97
732
media_image2.png
Greyscale
TAO also discloses displaying the captured information (para 135) for a display and the one or more viewpoints (para 52, 81, 92 and 158)
The examiner evidences BALLARD et al., US 20220217301 which discloses a TELECONFERENCING display including an autostereoscopic display (para 51-52) display 110 (Fig 1b). Ballard disclose a camera calibration system which may be preregistered or calibrated (preexisting) and update the calibration parameters and also use an autostereoscopic display (para 42), as well as multiple viewpoints (para 23, 27, 30, 34, 36 and 40)
The motivation to modify the above CLAVEU/TAO with BALLARD provides the conventional features of displaying captured images and multipleviewpoints, which may be an autostereoscopic (no 3d glasses required) or stereoscopie (with 3D glass) based upon the user’s/system needs and available resources as would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 6,
AS noted above in claim 1, CLAVEU discloses the target may be captured in multiple orientations/poses(positions) with respect to the overlapped fields of views in order to computer the reprojection error of the cameras with respect to the target and other cameras. Thus the capturing of a 3rd and 4th image would be met by a different orientation/pose of the target with respect to the 1st and 2nd camera and thus used to calculate the 2nd reprojection errors (para 80) of the respective cameras based upon a 2nd position/orientation of the calibration target.
In considering claim 7,
CLAVEU discloses the error is in pixels (para 29, 79, 113, 133, 149, 153-154)
TAO also discloses the error may be determined to be some number of pixels off the reference line.
The examiner notes the use of one or more pixels is an obvious modification to one of ordinary skill in the art wherein the smaller number of pixels used to compute an error is more precise/fine calibration and the larger number of pixels used as a threshold would be more of a coarse calibration in the combination above, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention as noted above.
In considering claim 8,
Refer to claim 1,
Regarding the processor (Fig 20, 80) and (memory 82) of CLAVEU.
In considering claim 13,
Refer to claim 6.
In considering claim 14,
Refer to claim 7.
In considering claim 15,
Refer to claim 1 and 8. (para 52-53, 85, 158-160, computer executable instructions).
In considering claim 18,
Refer to claim 4.
In considering claim 20,
Refer to claim 6.
In considering claim 22,
Based upon applications specification disclosing features as fiducials, the claims is rejected as follows:
CLAVEU discloses that the fiducials on the target pattern are used (para 5, 58, 85-87, 89, 108, 113, 121, 131, 142) to determine the position and/or orientation of the features (fiducials) using the captured images from the cameras and the calibration data (intrinsic/extrinsic) from each camera are used to calibrate the cameras and determine the pose/orientation of the calibration target 20, where the cameras can capture multiple image(s) of the target whether the target is repositioned/moved or not in order to ensure reprojection errors of the cameras meets a quality standard.
In considering claim 24,
Refer to claim 22.
In considering claim 26,
Refer to claim 22.
In considering claim 27,
The examiner notes the application used in CLAVEU/TAO does not disclose the camera being coupled to a display (the instant disclosure refers to a telepresence/teleconference/mirror display), however it is notoriously well-known in the art to couple cameras to a display, notably for a teleconferencing application, where such cameras are used to provide captured data of the user(s) to others.
The examiner incorporates BALLARD (Fig 1b) where a teleconferencing system including a display 110, includes (para 29) camera coupled to the display as shown, or can also be used as a virtual mirror (para 27, 50). Additionally BALLARD discloses the display may be sent to a remote device 102 for display (Fig 1a), where it is known when performing videoconferencing this typically includes at least 2 displays when performing a conference, which can be coupled (as disclosed via internet coupled; connected via wirelessly, wired including local or remotely).
The motivation for modifying the above combination with BALLARD would provide the user the ability to use an application like videoconferencing and/or virtual mirror application, where cameras provide the ability to capture data to relay to participant(s) included in such conference/virtual application thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 29,
Refer to claim 27.
In considering claim 33,
CLAVEAU disclose the reprojection error for the extrinsic paramaters (para 70, 121, 122, 142, 143, 145, 146, 149, 153-154) using quality criterion or threshold (para 108, 131) including overlapped pairs of views (para 32, 64, 77, 95, 125, 126, 128, 129)
TAO discloses an error threshold (para 91, 101, 104) and reprojection error (para 97, 99, 102.
Ballard discloses (para 37, 38 and 47) aligning the camera within the virtual field of view to ensure alignment between the cameras where misalignment between the cameras would negatively impact the image displayed, including overlapping viewpoints from pairs of cameras (para 30).
Thus given the combination above, discloses adjusting the extrinsic (camera pose/position/orientation) based upon the errors to ensure proper alignment to ensure proper calibration/display thus being an obvious combination to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 4, 11 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over CLAVEU et al., US 20170287166 and TAO et al., US 20220164987, BALLARD et al., US 20220217301 in view of ALEXOVSKI et al, US 20220067971
In considering claims 4, 11 and 18.
As disclosed by CLAVEU, the target may take on different positions, including poses and orientations (para 20, 49, 85, 101, 109, 117, 127, 136).
Regarding the newly amended limitation regarding determining deformation of the target, the examiner notes the combination above does not explicitly recite such.
The examiner newly evidences ALEXOVSKI et al, US 20220067971 (para 65) which discloses that a deformed target can be identified via calculation when performing camera calibration.
The motivation to modify the above combination ALEXOVSKI would ensure a proper calibration in the event the calibration target/fiducial became distorted and thus would ensure proper calibration by taking into account any deformities with such target thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
.
Claim(s) 28 and 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over CLAVEU et al., US 20170287166 and TAO et al., US 20220164987, BALLARD et al., US 20220217301 in view of GROSSMAN, US 20130182083.
In considering claim 28,
The combination of CLAVEU/TAO with Ballard discloses a camera coupled to a display, additionally Ballard also discloses that the display may include a “virtual mirror” applications or the use of a regular mirror (para 18, 27, 50).
The examiner BALLARD discloses videoconferencing which may display the image locally and/or remotely, where videoconferencing typically involves capturing an image of each participant for viewing via a coupled connection (wired or wireless or internet as disclosed).
Regarding the camera and 2nd display, which appears to be the local display, the examiner notes Grossman (Fig 4, para 69) discloses a display, which includes cameras which are not in the FOV of the display, but can captured the display contents via a mirror 40, to ensure proper calibration of the autostereoscopic display system.
The motivation to modify the above combination with Grossman would ensure any display used would be properly calibration, whether the cameras was in the FOV or not of the display, thus being an obvious modification to one of ordinary skill in the art before the effective filing date of the claimed invention.
In considering claim 30,
Refer to claim 28.
The examiner notes the applicant has amended the claims to claim what could/does happen and the claims recite “what” is occurring, however, the examiner suggest any subject matter disclosed which includes “how” the system/calibration is carried out if unique from the prior art is suggested in order to expedite prosecution.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brian Yenke whose telephone number is (571)272-7359. The examiner work schedule is Monday-Thursday, 0730-1830 hrs.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s Supervisor, John Miller, can be reached at (571)272-7353.
Any response to this action should be mailed to:
Commissioner of Patents and Trademarks
Washington, D.C. 20231
or faxed to:
(571)-273-8300
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the Technology Center 2400 Customer Service Office whose telephone number is
(703)305-HELP.
General information about patents, trademarks, products and services offered by the United States Patent and Trademark Office (USPTO), and other related information is available by contacting the USPTO’s General Information Services Division at:
800-PTO-9199 or 703-308-HELP
(FAX) 703-305-7786
(TDD) 703-305-7785
An automated message system is available 7 days a week, 24 hours a day providing informational responses to frequently asked questions and the ability to order certain documents. Customer service representatives are available to answer questions, send materials or connect customers with other offices of the USPTO from 8:30 a.m. - 8:00p.m. EST/EDT, Monday-Friday excluding federal holidays.
For other technical patent information needs, the Patent Assistance Center can be reached through customer service representatives at the above numbers, Monday through Friday (except federal holidays) from 8:30 a.m. to 5:00 p.m. EST/EDT.
The Patent Electronic Business Center (EBC) allows USPTO customers to retrieve data, check the status of pending actions, and submit information and applications. The tools currently available in the Patent EBC are Patent Application Information Retrieval (PAIR) and the Electronic Filing System (EFS). PAIR (http://pair.uspto.gov) provides customers direct secure access to their own patent application status information, as well as to general patent information publicly available. EFS allows customers to electronically file patent application documents securely via the Internet. EFS is a system for submitting new utility patent applications and pre-grant publication submissions in electronic publication-ready form. EFS includes software to help customers prepare submissions in extensible Markup Language (XML) format and to assemble the various parts of the application as an electronic submission package. EFS also allows the submission of Computer Readable Format (CRF) sequence listings for pending biotechnology patent applications, which were filed in paper form.
/BRIAN P YENKE/Primary Examiner, Art Unit 2422
Prosecution Insights