DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The Information Disclosure Statement filed 08/15/2024 has been considered by examiner.
Specification
The abstract of the disclosure is objected to because of its length and verbose nature. Applicant is reminded of the proper language and format for an abstract of the disclosure. The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details. The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a raycasting execution unit [generic place holder] configured to perform raycasting on a three-dimensional space region comprising a plurality of photographing images on the basis of an origin point [functional language without structure];
an overlap region determination unit [generic place holder] configured to determine whether an overlap region that is a region that overlaps between the photographing images is present, by determining whether intersection points that are generated as any one virtual line that is generated by performing the raycasting and planes of the photographing images are intersected are at least two [functional language without structure];
a blending region generation unit [generic place holder] configured to generate a blending region that is a blending target region between the photographing images, based on the overlap region [functional language without structure];
a gradient correction unit [generic place holder] configured to correct gradients of a first image plane and a second image plane corresponding to the blending region, on the basis of at least one plane intersection point that is formed as the first image plane and the second image plane that are the planes of the photographing images are intersected [functional language without structure]; and
an image matching unit [generic place holder] configured to generate a panoramic image by matching the at least one photographing image [functional language without structure]” in claim 9.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. No corresponding structure has been found in the specification.
If applicant does not intend to have these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 9-15 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 9 recites 112(f) language without supporting structures present in the specification. Claims 10-15 are dependent on claim 9, and accordingly are also rejected.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 9 contains limitations that invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
a raycasting execution unit [generic place holder] configured to perform raycasting on a three-dimensional space region comprising a plurality of photographing images on the basis of an origin point [functional language without structure];
an overlap region determination unit [generic place holder] configured to determine whether an overlap region that is a region that overlaps between the photographing images is present, by determining whether intersection points that are generated as any one virtual line that is generated by performing the raycasting and planes of the photographing images are intersected are at least two [functional language without structure];
a blending region generation unit [generic place holder] configured to generate a blending region that is a blending target region between the photographing images, based on the overlap region [functional language without structure];
a gradient correction unit [generic place holder] configured to correct gradients of a first image plane and a second image plane corresponding to the blending region, on the basis of at least one plane intersection point that is formed as the first image plane and the second image plane that are the planes of the photographing images are intersected [functional language without structure]; and
an image matching unit [generic place holder] configured to generate a panoramic image by matching the at least one photographing image [functional language without structure]
However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. No corresponding structures have been found within the specification. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. Additionally, claims 10-15 are dependent on claim 9, and accordingly are also rejected.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
14. If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 9 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Khwaja (US 11049218 B2).
With respect to claim 1, Khwaja teaches a panoramic image blending method (see figure 2 element 230) using a panoramic image blending apparatus that performs image matching based on a plurality of photographing images (see figure 2 element 210), the panoramic image blending method comprising:
a raycasting execution step of performing raycasting on a three-dimensional space region comprising a plurality of photographing images on the basis of an origin point (see figure 4 with center of camera as orgin);
an overlap region determination step of determining whether an overlap region that is a region that overlaps between the photographing images is present, by determining whether intersection points that are generated as any one virtual line that is generated by performing the raycasting and planes of the photographing images are intersected are at least two (see figure 4 element 216 and “In particular embodiments, and as described below, angular overlap 216 may be used to identify image features and create a stitched image that seamlessly shows an entire view as captured by camera system 210. Although this disclosure describes and illustrates particular cameras having particular angular overlaps, this disclosure contemplates any suitable cameras having any suitable angular overlaps.” Page 46 col 11 lines 5-12);
a blending region generation step of generating a blending region that is a blending target region between the photographing images (see figure 2 element 230 and “One technical improvement of the stitching techniques discussed herein is to generate a seamless smooth image and remove/minimize these image artifacts, as shown for example by reference numerals 212b and 214b in FIG. 1.” Page 42 col 3 lines 3-6), based on the overlap region (“In particular embodiments, and as described below, angular overlap 216 may be used to identify image features and create a stitched image that seamlessly shows an entire view as captured by camera system 210. Although this disclosure describes and illustrates particular cameras having particular angular overlaps, this disclosure contemplates any suitable cameras having any suitable angular overlaps.” Page 46 col 11 lines 5-12);
a gradient correction step of correcting gradients of a first image plane and a second image plane corresponding to the blending region (“One technical improvement of the stitching techniques discussed herein is to generate a seamless smooth image and remove/minimize these image artifacts, as shown for example by reference numerals 212b and 214b in FIG. 1. An example stitched image generated using an example technique discussed herein is shown in FIG. 27. In order to stitch two images, the images may be first warped onto a sphere and defished to produce rectilinear images. Edge detection may be performed on these images using sobel gradients to calculate image edges and contrast lines (see for example FIG. 23) such that images can be compared to each other. Grid optimization may be performed to minimize the difference between the normalized gradients of overlapping regions of the two images.” Page 42 col 3 lines 3-16), on the basis of at least one plane intersection point that is formed as the first image plane and the second image plane that are the planes of the photographing images are intersected (see figure 4 element 216 and “In particular embodiments, and as described below, angular overlap 216 may be used to identify image features and create a stitched image that seamlessly shows an entire view as captured by camera system 210. Although this disclosure describes and illustrates particular cameras having particular angular overlaps, this disclosure contemplates any suitable cameras having any suitable angular overlaps.” Page 46 col 11 lines 5-12); and
an image matching step of generating a panoramic image by matching the at least one photographing image (“This process may be repeated until the difference between the normalized gradients of overlapping regions of the two images is within or less than a certain threshold. Once the difference is within the threshold (e.g., edges or edge lines of two images overlap with one another), seam estimation and blending may be performed to produce a combined or stitched image, as shown for example in FIG. 27.” Page 42 col 3 lines 43-50).
With respect to claim 9, Khwaja teaches all claim limitations in consideration of claim 1, due to the substantial similarity of claims 1 and 9, with claim 9 being directed towards a generic representation of an apparatus that is configured to execute the method of claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-8, 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Khwaja, in view of Chuang (US 9288392 B2).
With respect to claim 2, Khwaja teaches the panoramic image blending method of claim 1, and further teaches wherein: the blending region comprises a first blending region that is a region of the blending region in which the at least one virtual line and the first image plane are first intersected (see figure 4 element 216 and “In particular embodiments, and as described below, angular overlap 216 may be used to identify image features and create a stitched image that seamlessly shows an entire view as captured by camera system 210. Although this disclosure describes and illustrates particular cameras having particular angular overlaps, this disclosure contemplates any suitable cameras having any suitable angular overlaps.” Page 46 col 11 lines 5-12, the region in the overlap region corresponding to an image from a first camera) and a second blending region that is a region of the blending region in which the at least one virtual line and the second image plane are first intersected (see figure 4 element 216 and “In particular embodiments, and as described below, angular overlap 216 may be used to identify image features and create a stitched image that seamlessly shows an entire view as captured by camera system 210. Although this disclosure describes and illustrates particular cameras having particular angular overlaps, this disclosure contemplates any suitable cameras having any suitable angular overlaps.” Page 46 col 11 lines 5-12, the region in the overlap region corresponding to an image from a second camera), however Khwaja does not teach, in the gradient correction step, the gradient of the first image plane corresponding to the first blending region is corrected to be greater than the gradient of the second image plane corresponding to the first blending region, and the gradient of the first image plane corresponding to the second blending region is corrected to be greater than the gradient of the second image plane corresponding to the second blending region.
Chuang teaches in the gradient correction step, the gradient of the first image plane corresponding to the first blending region is corrected to be greater than the gradient of the second image plane corresponding to the first blending region (see figure 6), and the gradient of the first image plane corresponding to the second blending region is corrected to be greater than the gradient of the second image plane corresponding to the second blending region (see figure 6).
Chuang is analogous art in the same field of endeavor as the claimed invention. Chuang is directed towards image blending and gradient correction (“Accordingly, the invention is directed to an image capturing device and an image processing method thereof, by which a main object of an image is determined according to images captured with different focal lengths, so as to generate an image with a clear main object and a natural bokeh effect. On the other hand, in the image processing method, a ghost phenomenon occurred when generating a full depth of field (DOF) image is avoided according to the images captured with different focal lengths.” Page 10 lines 10-19). A person of ordinary skill would have found it obvious to combine the teaching of Khwaja and Chuang by utilizing Chuang’s gradient correction step in place of Khwaja’s, with the expectation that doing so would lead to a more satisfactory final image (“Accordingly, the invention is directed to an image capturing device and an image processing method thereof, by which a main object of an image is determined according to images captured with different focal lengths, so as to generate an image with a clear main object and a natural bokeh effect. On the other hand, in the image processing method, a ghost phenomenon occurred when generating a full depth of field (DOF) image is avoided according to the images captured with different focal lengths.” Page 10 lines 10-19).
With respect to claim 3, Khwaja and Chuang teach the panoramic image blending method of claim 2. Chuang further teaches wherein in the gradient correction step,
the gradient of the first image plane corresponding to the first blending region is corrected to be increased as the gradient of the first image plane becomes distant in a first direction from the plane intersection point (see figure 5 and figure 6 and “A first image and a second image are captured with a first focal length and a second focal length, where the first focal length is focused on at least one main object. A geometric calibration procedure is performed on the second image to produce the motion calibrated second image. A gradient operation is performed on each pixel of the first image to produce a plurality of first gradients, and the gradient operation is performed on each pixel of the motion calibrated second image to produce a plurality of second gradients. Each of the first gradients and the corresponding second gradient are compared to generate a plurality of first pixel comparison results, and a first parameter map is generated according to the first pixel comparison results. A blending image is produced according to the first parameter map and the first image, and an output image is produced at least in accordance with the blending image.” Page 10 col 2 lines 23-37),
the gradient of the second image plane corresponding to the second blending region is corrected to be increased as the gradient of the second image plane becomes distant in a second direction from the plane intersection point (see figure 5 and figure 6 “A first image and a second image are captured with a first focal length and a second focal length, where the first focal length is focused on at least one main object. A geometric calibration procedure is performed on the second image to produce the motion calibrated second image. A gradient operation is performed on each pixel of the first image to produce a plurality of first gradients, and the gradient operation is performed on each pixel of the motion calibrated second image to produce a plurality of second gradients. Each of the first gradients and the corresponding second gradient are compared to generate a plurality of first pixel comparison results, and a first parameter map is generated according to the first pixel comparison results. A blending image is produced according to the first parameter map and the first image, and an output image is produced at least in accordance with the blending image.” Page 10 col 2 lines 23-37), and
the first direction is a direction from a central point of the second image plane to a central point of the first image plane (“Then, in step S540, the gradient calculating module 440 performs a gradient operation on each pixel of the first image Img1 to produce a plurality of first gradients G1, and performs the gradient operation on each pixel of the motion calibrated second image Img2_cal to produce a plurality of second gradients G2. The gradient operation can be a horizontal gradient operation, a vertical gradient operation or dual diagonal gradient operations, which is not limited by the invention. Namely, the first gradient and the second gradient can be a horizontal gradient, a vertical gradient or dual diagonal gradients according to the method of the gradient operation.” Page 13 col 8 lines 48-59), and the second direction is a direction from the central point of the first image plane to the central point of the second image plane (“Then, in step S540, the gradient calculating module 440 performs a gradient operation on each pixel of the first image Img1 to produce a plurality of first gradients G1, and performs the gradient operation on each pixel of the motion calibrated second image Img2_cal to produce a plurality of second gradients G2. The gradient operation can be a horizontal gradient operation, a vertical gradient operation or dual diagonal gradient operations, which is not limited by the invention. Namely, the first gradient and the second gradient can be a horizontal gradient, a vertical gradient or dual diagonal gradients according to the method of the gradient operation.” Page 13 col 8 lines 48-59).
With respect to claim 4, Khwaja and Chuang teach the panoramic image blending method of claim 3. Chuang further teaches wherein in the gradient correction step,
the gradient of the intersection point of the second image plane corresponding to the first blending region, which intersects the same virtual line as the intersection point of the first image plane corresponding to the first blending region, is corrected to be inversely proportional to the gradient of the intersection point of the first image plane corresponding to the first blending region (“It should be noticed that in the present embodiment, since the first image Img1 is captured by focusing on the main object, compared to the motion calibrated image Img2_cal, the main object in the first image Img1 is clearer. Namely, the gradient of the pixel in the main object area of the first image Img1 is greater than the gradient of the pixel located at the same position in the motion calibrated second image Img2_cal. Conversely, since the motion calibrated second image Img2_cal is captured by focusing on the background, the gradient of the pixel in the background area of the first image Img1 is smaller than the gradient of the pixel located at the same position in the motion calibrated second image Img2_cal.” Pages 13-14 cols 8 (lines 66-67) – 9 (lines 1-11) And figure 5), and
the gradient of the intersection point of the second image plane corresponding to the second blending region, which intersects the same virtual line as the intersection point of the first image plane corresponding to the second blending region, is corrected to be inversely proportional to the gradient of the intersection point of the first image plane corresponding to the second blending region (“It should be noticed that in the present embodiment, since the first image Img1 is captured by focusing on the main object, compared to the motion calibrated image Img2_cal, the main object in the first image Img1 is clearer. Namely, the gradient of the pixel in the main object area of the first image Img1 is greater than the gradient of the pixel located at the same position in the motion calibrated second image Img2_cal. Conversely, since the motion calibrated second image Img2_cal is captured by focusing on the background, the gradient of the pixel in the background area of the first image Img1 is smaller than the gradient of the pixel located at the same position in the motion calibrated second image Img2_cal.” Pages 13-14 cols 8 (lines 66-67) – 9 (lines 1-11) And figure 5).
With respect to claim 5, Khwaja and Chuang teach the panoramic image blending method of claim 4. Chuang teaches it further comprising a plane distance measuring step of measuring a plane distance that is a distance between a first intersection point of the first image plane and a second intersection point of the second image plane, which intersect the same virtual line that extends from the origin point (“In the aforementioned embodiment, the second focal length is, for example, focused on the background, and a background blur image with a blurry background and clear main object is produced. According to the description of FIG. 3, it is known that the image processing method of the invention may obtain the final output image according to a plurality of images. In this way, in other embodiments, when the image capturing device captures another image with a third focal length focused on the foreground, the image capturing device can produce an image with blurry foreground and background and clear main object through calculation by using the aforementioned background blur image and the image captured by focusing on the foreground according to a process the same with that used for producing the background blur image.” Page 15 col 12 lines 3-18),
wherein the gradient correction step comprises correcting gradients of the first intersection point of the first image plane and the second intersection point of the second image plane corresponding to the blending region, based on the plane distance (see figure 3 and figure 8 elements 830, 840, 850, and 860 and “Similarly, according to related description of FIG. 3, it is known that the image processing method of the present embodiment can obtain the final output image according to a plurality of images. Therefore, in the present embodiment, the image capturing device 800 may capture a plurality of images with a plurality of different focal lengths, and blend the images captured with different focal lengths to produce a clear full DOF image. In an actual application, the scene is first analysed to determine the number of images of different focal lengths that are required for producing the entirely clear full DOF image.” Page 17 col 15 lines 42-54).
With respect to claim 6, Khwaja and Chuang teach the panoramic image blending method of claim 5. Chuang further teaches wherein: the plane distance measuring step comprises determining a maximum plane distance between the intersection points of the first image plane and the second image plane that intersect the same virtual line in the blending region (“Therefore, in the present embodiment, the map generating module 440 generates the parameter map according to comparison results of the gradients of the pixels in the first image Img1 and the motion calibrated second image Img2_cal. In other words, the parameter map carries comparison result information of the gradients of the pixels located at the same position in the first image Img1 and the motion calibrated second image Img2_cal. In this way, the image capturing device 800 can learn whether a pixel of a certain position is located at a clear part within the first focal length in the first image Img1 or located at a clear part within the second focal length in the second image Imge2 according to the parameter map. In this way, the image blending module 850 can blend the clear parts of the two images for produce an output image with more clear parts.” Page 16 col 13 lunes 11-25), and
the gradient correction step comprises calculating a distance ratio that is a ratio of the maximum plane distance and the plane distance and correcting the gradients of the first intersection point of the first image plane and the second intersection point of the second image plane corresponding to the blending region, based on the distance ratio (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10 and “Generally, pixels located at a same position in two images have different gradients, i.e. the aforementioned first gradient G1 and the second gradient G2. On the other hand, regarding the pixel of the same position, if the pixel of such position has a higher gradient in the first image (i.e. G1 is greater than G2), it represents that the pixel of such position is located at a clearer area of the first image (i.e. an area within the first focal length). If the pixel of such position has a higher gradient in the second image (i.e. G2 is greater than G1), it represents that the pixel of such position is located at a clearer area of the second image (i.e. an area within the second focal length). Namely, the map generating module 840 can obtain the parameter map according to the pseudo code (1), though the invention is not limited thereto” pages 15-16 cols 12 (lines 64-66) – 13 (lines 1-10)).
With respect to claim 7, Khwaja and Chuang teach the panoramic image blending method of claim 5. Chuang further teaches wherein: the blending region generation step comprises comparing preset distance information that is condition information on which the blending region is formed and the plane distance, (“Similarly, according to related description of FIG. 3, it is known that the image processing method of the present embodiment can obtain the final output image according to a plurality of images. Therefore, in the present embodiment, the image capturing device 800 may capture a plurality of images with a plurality of different focal lengths, and blend the images captured with different focal lengths to produce a clear full DOF image. In an actual application, the scene is first analysed to determine the number of images of different focal lengths that are required for producing the entirely clear full DOF image.” Page 17 col 15 lines 43-53) and generating the blending region based on the overlap region in which the plane distance is equal to or smaller than preset distance information (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10 and “Generally, pixels located at a same position in two images have different gradients, i.e. the aforementioned first gradient G1 and the second gradient G2. On the other hand, regarding the pixel of the same position, if the pixel of such position has a higher gradient in the first image (i.e. G1 is greater than G2), it represents that the pixel of such position is located at a clearer area of the first image (i.e. an area within the first focal length). If the pixel of such position has a higher gradient in the second image (i.e. G2 is greater than G1), it represents that the pixel of such position is located at a clearer area of the second image (i.e. an area within the second focal length). Namely, the map generating module 840 can obtain the parameter map according to the pseudo code (1), though the invention is not limited thereto” pages 15-16 cols 12 (lines 64-66) – 13 (lines 1-10)), and
the gradient correction step comprises calculating a distance ratio that is a ratio of the preset distance information and the plane distance (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10, second map value), and correcting the gradients of the first intersection point of the first image plane and the second intersection point of the second image plane corresponding to the blending region, based on the distance ratio (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10, second map value).
With respect to claim 8, Khwaja and Chuang teach the panoramic image blending method of claim 3. Chuang further teaches wherein in the gradient correction step, the gradient of the first image plane corresponding to the first blending region is corrected to be increased as the gradient of the first image plane becomes close to the central point of the first image plane from the plane intersection point (“The map generating module 440 determines whether the gradient comparison value of each position is greater than a first gradient threshold (step S621). If the gradient comparison value is greater than the first gradient threshold, the map generating module 440 sets the parameter corresponding to the gradient comparison value to a first value (step S622), and the first value is referred to as a bokeh background value. In other words, if the gradient comparison value is greater than the first gradient threshold, it represents that the pixel of such position is located in the background area. If the gradient comparison value is not greater than the first gradient threshold, the map generating module 440 determines whether the gradient comparison value is greater than a second gradient threshold (step S623). If the gradient comparison value is greater than the second gradient threshold, the map generating module 440 sets the parameter corresponding to the gradient comparison value to a second value (step S624), and the second value is referred to as a bokeh edge value. In brief, if the gradient comparison value is between the second gradient threshold and the first gradient threshold, it represents that the pixel of such position is located in an edge area connected between the main object area and the background area. If the gradient comparison value is not greater than the second gradient threshold, the map generating module 440 sets the parameter corresponding to the gradient comparison value to a third value (step S625), and the third value is referred to as a bokeh main object value, i.e. the pixel of such position is located in the main object area.” Page 14 cols 9 (lines 64-67) and 10 (lines 1-24)), and
the gradient of the second image plane corresponding to the second blending region is corrected to be increased as the gradient of the second image plane becomes close to the central point of the second image plane from the plane intersection point (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10 and figure 6).
With respect to claim 10, Khwaja and Chuang teach the panoramic image blending apparatus of claim 9 and all additional claim limitations in consideration of claim 2, due to the substantial similarities between claims 10 and 2, with claim 10 being directed towards a generically recited apparatus that is configured to execute claim 2.
With respect to claim 11, Khwaja and Chuang teach the panoramic image blending apparatus of claim 10 and all additional claim limitations in consideration of claim 3, due to the substantial similarities between claims 11 and 3, with claim 11 being directed towards a generically recited apparatus that is configured to execute claim 3.
With respect to claim 12, Khwaja and Chuang teach the panoramic image blending apparatus of claim 11. Chuang teaches it further comprising a plane distance measuring unit of measuring a plane distance that is a distance between a first intersection point of the first image plane and a second intersection point of the second image plane, which intersect the same virtual line that extends from the origin point (“In the aforementioned embodiment, the second focal length is, for example, focused on the background, and a background blur image with a blurry background and clear main object is produced. According to the description of FIG. 3, it is known that the image processing method of the invention may obtain the final output image according to a plurality of images. In this way, in other embodiments, when the image capturing device captures another image with a third focal length focused on the foreground, the image capturing device can produce an image with blurry foreground and background and clear main object through calculation by using the aforementioned background blur image and the image captured by focusing on the foreground according to a process the same with that used for producing the background blur image.” Page 15 col 12 lines 3-18),
wherein the gradient correction unit comprises correcting gradients of the first intersection point of the first image plane and the second intersection point of the second image plane corresponding to the blending region, based on the plane distance (see figure 3 and figure 8 elements 830, 840, 850, and 860 and “Similarly, according to related description of FIG. 3, it is known that the image processing method of the present embodiment can obtain the final output image according to a plurality of images. Therefore, in the present embodiment, the image capturing device 800 may capture a plurality of images with a plurality of different focal lengths, and blend the images captured with different focal lengths to produce a clear full DOF image. In an actual application, the scene is first analysed to determine the number of images of different focal lengths that are required for producing the entirely clear full DOF image.” Page 17 col 15 lines 42-54).
With respect to claim 13, Khwaja and Chuang teach the panoramic image blending apparatus of claim 12. Chuang further teaches wherein: the plane distance measuring unit comprises determining a maximum plane distance between the intersection points of the first image plane and the second image plane that intersect the same virtual line in the blending region (“Therefore, in the present embodiment, the map generating module 440 generates the parameter map according to comparison results of the gradients of the pixels in the first image Img1 and the motion calibrated second image Img2_cal. In other words, the parameter map carries comparison result information of the gradients of the pixels located at the same position in the first image Img1 and the motion calibrated second image Img2_cal. In this way, the image capturing device 800 can learn whether a pixel of a certain position is located at a clear part within the first focal length in the first image Img1 or located at a clear part within the second focal length in the second image Imge2 according to the parameter map. In this way, the image blending module 850 can blend the clear parts of the two images for produce an output image with more clear parts.” Page 16 col 13 lunes 11-25), and
the gradient correction unit comprises calculating a distance ratio that is a ratio of the maximum plane distance and the plane distance and correcting the gradients of the first intersection point of the first image plane and the second intersection point of the second image plane corresponding to the blending region, based on the distance ratio (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10 and “Generally, pixels located at a same position in two images have different gradients, i.e. the aforementioned first gradient G1 and the second gradient G2. On the other hand, regarding the pixel of the same position, if the pixel of such position has a higher gradient in the first image (i.e. G1 is greater than G2), it represents that the pixel of such position is located at a clearer area of the first image (i.e. an area within the first focal length). If the pixel of such position has a higher gradient in the second image (i.e. G2 is greater than G1), it represents that the pixel of such position is located at a clearer area of the second image (i.e. an area within the second focal length). Namely, the map generating module 840 can obtain the parameter map according to the pseudo code (1), though the invention is not limited thereto” pages 15-16 cols 12 (lines 64-66) – 13 (lines 1-10)).
With respect to claim 14, Khwaja and Chuang teach the panoramic image blending apparatus of claim 12. Chuang further teaches wherein: the blending region generation unit comprises comparing preset distance information that is condition information on which the blending region is formed and the plane distance, (“Similarly, according to related description of FIG. 3, it is known that the image processing method of the present embodiment can obtain the final output image according to a plurality of images. Therefore, in the present embodiment, the image capturing device 800 may capture a plurality of images with a plurality of different focal lengths, and blend the images captured with different focal lengths to produce a clear full DOF image. In an actual application, the scene is first analysed to determine the number of images of different focal lengths that are required for producing the entirely clear full DOF image.” Page 17 col 15 lines 43-53) and generating the blending region based on the overlap region in which the plane distance is equal to or smaller than preset distance information (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10 and “Generally, pixels located at a same position in two images have different gradients, i.e. the aforementioned first gradient G1 and the second gradient G2. On the other hand, regarding the pixel of the same position, if the pixel of such position has a higher gradient in the first image (i.e. G1 is greater than G2), it represents that the pixel of such position is located at a clearer area of the first image (i.e. an area within the first focal length). If the pixel of such position has a higher gradient in the second image (i.e. G2 is greater than G1), it represents that the pixel of such position is located at a clearer area of the second image (i.e. an area within the second focal length). Namely, the map generating module 840 can obtain the parameter map according to the pseudo code (1), though the invention is not limited thereto” pages 15-16 cols 12 (lines 64-66) – 13 (lines 1-10)), and
the gradient correction unit comprises calculating a distance ratio that is a ratio of the preset distance information and the plane distance (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10, second map value), and correcting the gradients of the first intersection point of the first image plane and the second intersection point of the second image plane corresponding to the blending region, based on the distance ratio (
PNG
media_image1.png
183
292
media_image1.png
Greyscale
pseudo code (1) page 14 col 10, second map value).
With respect to claim 15, Khwaja and Chuang teach the panoramic image blending apparatus of claim 11 and all additional claim limitations in consideration of claim 8, due to the substantial similarities between claims 15 and 8, with claim 15 being directed towards a generically recited apparatus that is configured to execute claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REBECCA C WILLIAMS whose telephone number is (571)272-7074. The examiner can normally be reached M-F 7:30am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew W Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REBECCA COLETTE WILLIAMS/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights