Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This communication is a non-Final office action in merits. Claims 1-9, 11-21, after preliminary amendment, are presently pending and have been elected and considered below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 3/31/2025, 4/8/2025, and 2/18/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN202211595183.7, filed on 12/13/2022.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 5-9, 11-13, 16-18, 21 are rejected under 35 U.S.C. 102(2) as anticipated by or, in the alternative, under 35 U.S.C. 103 as obvious over US 2021/0082086 A1, Bichu et al. (hereinafter Bichu).
As to claim 1, Bichu discloses a panoramic image processing method, comprising: acquiring a first image captured by a first image acquisition apparatus and a second image captured by a second image acquisition apparatus (Figs 2, 11; pars 0002-0004, 0022, 0043, acquiring images captured by cameras), and respectively converting the first image and the second image into a first panoramic image and a second panoramic image (Figs 2-4; pars 0002-0003, 0006-0007, 0067, generating panoramic images);
determining parallax information between the first panoramic image and the second panoramic image (Figs 3B, 19, 24, determining a first and second disparity maps (e.g. parallax information); pars 0006-0007, 0044-0052, 0054-0055, 0061), and
generating depth information corresponding to the parallax information (Figs 3A, 16, 27; pars 0027, 0043-0049, 0058, 0071, estimating a depth of objects in a stitching region of two images); and
based on the depth information, mapping the first image and the second image into a same device space, and according to a mapping result in the same device space, generating a stitched panoramic image of the first image and the second image (Figs 2, 3B, 18-19, 26-27; pars 0043-0050, 0058, 0071, images/frames being mapped, aligned, then stitched).
As to claim 2, Bichu discloses the method according to claim 1, wherein determining the parallax information between the first panoramic image and the second panoramic image comprises: performing epipolar rectification on the first panoramic image and the second panoramic image (Fig 17; pars 0028, 0197, 0200-0203, performing epipolar rectification), and for the two panoramic images after the epipolar rectification, estimating a parallax between the two panoramic images using a preset stereo matching network, to generate the parallax information between the first panoramic image and the second panoramic image (pars 0058-0059, 0100, 0114, 0195, 0200-0201, 0205-0206, once the images being rectified, key points of both images being matched and a reduced parallax can be estimated in the device space/platform).
As to claim 5, Bichu discloses the method according to claim 1, wherein after generating the depth information corresponding to the parallax information, the method further comprises: conducting statistics of depth information of adjacent multi-frame images (Fig 27; pars 0058, 0071-0074, 0107, 0124, estimating the depth of objects and determining subsequent disparity map for each pixel associated with subsequent frames (representing statistical depth information in adjacent frames)) and determining, based on a statistical result, whether to generate smoothed depth information (pars 0065, adjusting each of the filtered disparity maps if needed); and if the smoothed depth information is generated, replacing the depth information with the smoothed depth information (pars 0211-0212, 0215, 0218, 0230, 0236, 0238, improved smoothness being obtained in the stitched output (e.g. replacing unsmoothed one)).
As to claim 6, Bichu discloses the method according to claim 5, wherein determining, based on the statistical result, whether to generate the smoothed depth information comprises: for a pixel point at a specified position in the adjacent multi-frame images, identifying a depth value of the pixel point in respective frames of images from the statistical result, and if there is a jump value among the identified depth values, generating the smoothed depth information for the pixel point (Fig 27; pars 0058, 0071-0074, 0107, 0124, also see rejection in claim 5).
As to claim 7, Bichu discloses the method according to claim 6, wherein generating the smoothed depth information for the pixel point comprises: removing the jump value from the identified depth values (pars 0060, 0066, 0215, 0283, removing undesired artifacts and/or discontinuity), performing an interpolation operation on remaining depth values, and taking a result of the interpolation operation as a smoothed depth value (pars 0062, 0103-0104, 0110-0111, 0218).
As to claim 8, Bichu discloses the method according to claim 1, wherein converting the first image into the first panoramic image comprises: acquiring a calibration coefficient of the first image acquisition apparatus, the calibration coefficient at least comprising an internal parameter and a distortion coefficient (pars 0004-0007, 0052, 0055, 0089, 0102, 0115, 0124, parameters/coefficients with respect to distortion/misalignment); through the distortion coefficient, performing distortion correction on pixel points in the first image, to obtain the corrected pixel points (pars 0069, 0088, 0195, 0288-0289, 0291, correction in pixel level); and through the internal parameter, mapping the corrected pixel points into the first panoramic image, to generate pixel coordinates of the corrected pixel points in the first panoramic image (Fig 22; pars 0004-0007, 0061-0063, 0125, 0129, 0137, 0141, 0168, 0198, 0210, 0243, 0246-0252, 0277, 0283, 0285).
As to claim 9, it is rejected with the same reason as set forth in claim 8, as the same conversion capabilities and features in claim 8 can be applied to claim 9.
10. (Cancelled)
As to claim 11, it is a device claim encompassed claim 1. Rejection of claim 1 is therefore incorporated herein.
As to claim 12, it recites a non-transitory CRM storing a computer program executed to perform functions and features of claim 1. Rejection of claim 1 is therefore incorporated herein.
As to claim 13, it is rejected with the same reason as set forth in claim 2.
As to claims 16-17, they are rejected with the same reason as set forth in claims 5-6, respectively.
As to claim 18, t is rejected with the same reason as set forth in claim 2.
As to claim 21, they are rejected with the same reason as set forth in claim 5.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 3-4, 14-15, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bichu in view of CN 108257183, Geng (hereinafter Geng), see google translation for citations.
As to claim 3, Bichu discloses the method according to claim 1, wherein mapping the first image and the second image into the same device space comprises: inverse projecting pixel points in the first image to a first camera coordinate system of the first image acquisition apparatus, and inverse projecting pixel points in the second image to a second camera coordinate system of the second image acquisition apparatus (Figs 4A-4C, 5, 15; pars 0026, 0057, 0119, 0125-0127, projection or transformation between the camera coordinates and two dimensional image coordinate represented by pixel coordinates); determining rotation and translation information between the first camera coordinate system and the second camera coordinate system (pars 0118, 0121, 0125-0129, 0151-0153), and based on the depth information and the rotation and translation information, generating a coordinate transformation matrix between the first camera coordinate system and the second camera coordinate system (Figs 5, 17; pars 0126-0128, 0152, 0166-0167, 0181, 0197, transformation matrix being generated between ); and through the coordinate transformation matrix, mapping inverse projection results in the first camera coordinate system and the second camera coordinate system to a same camera coordinate system (pars 0089, 0167-0168, 0189, 0197, 0201, 0206).
Bichu does not expressly disclose inverse projection and mapping from the pixel coordination to camera coordination. Geng, in the same or similar field of endeavor, further teaches the inverse projection can be obtained that the coordinate values of the projection points corresponding to the coordinate values of the pixel points on the image acquired by the rear lens being multiplied by the inverse matrix of the rotation matrix in the external parameters (page 7, par 11).
Therefore, consider Bichu and Geng’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Geng’s teachings in Bichu’s method to improve alignment accuracy of the camera lens, effeteness of the panoramic image, as well as user experience.
As to claim 4, Bichu as modified discloses the method according to claim 1, wherein generating the stitched panoramic image of the first image and the second image comprises: projecting the mapping result in the same device space onto a surface of a virtual unit ball (Bichu: Fig 15; pars 0026, 0057, 0142, 0145, 0167-0169, both images being projected to a unit sphere; Geng: page 7, par 2, the unit sphere being a virtual unit sphere), and taking an image obtained by unfolding the surface of the virtual unit ball along a specified longitude line as the stitched panoramic image (Bichu: Fig 15; pars 0025, 0057; Geng: page 7, par 2).
As to claim 14, it is rejected with the same reason as set forth in claim 3.
As to claim 15, it is rejected with the same reason as set forth in claim 4.
As to claim 19, it is rejected with the same reason as set forth in claim 3.
As to claim 20 is rejected with the same reason as set forth in claim 4.
Examiner’s Note
Examiner has cited particular column, line number, paragraphs and/or figure(s) in the reference(s) as applied to the claims for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the reference(s) in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUN SHEN whose telephone number is (571)270-7927. The examiner can normally be reached on Mon-Fri 8:30-5:50 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUN SHEN/
Primary Examiner, Art Unit 2662
Prosecution Insights