DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on November 1st, 2024 and June 18th, 2025 has been considered and the listed references were noted.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: 420.
Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 21 objected to because of the following informalities:
“A non-transitory recording medium storing a computer program….”, although may be technically patent eligible, could be interpreted broadly compared to just computer storage storing a computer program. For example, a piece of paper could be considered a non-transitory recording medium as a computer program could be written or typed on it. The examiner recommends the following revisions to ensure the preamble of the claim to be more in line with the articles of manufacture and statutory categories: “A non-transitory computer-readable recording medium storing a computer program…”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “an imager to generate a captured image” in claims 1 and 3-21.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Status of Claims
Claim 2 is canceled, Claims 1 and 3-21 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Meng Lu (CN 112752026 A) in view of Maleki et al. (US 2022/0368881 w/ EFD of January 1st, 2020).
Regarding Claim 1, Lu discloses “An apparatus for image analysis, the apparatus comprising:” (Lu, Paragraph [0119] discloses: “As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, apparatus (device), or computer program product.”; Paragraph [0043] discloses “S302, obtaining first depth values of the interested areas of the first focused images obtained through a second camera of the terminal device, and obtaining a plurality of first depth values corresponding to the first focused images one by one. Wherein the first depth value is calculated from depth values of a plurality of different pixels of the region of interest of the first in-focus image.”; From [0043], we can see that this step in the invention obtains first depth values from images, which is used to further analyze the image to perform different actions, such as auto focusing which will be mentioned later); “at least one memory storing instructions, and at least one processor configured to execute the instructions, wherein, by executing the instructions, the at least one processor is configured to control:” (Lu, Paragraph [0113], discloses “the electronic device 60 provided in this embodiment may include a processor 601 and a memory 602, where the memory 602 stores program instructions, and the processor 601 is configured to call the program instructions in the memory 502 to execute an auto-focusing method provided in any method embodiment of the present application.”); “an imager to generate a captured image” (Lu, Paragraph [0036], discloses “s301, in the using process of the terminal device, acquiring a plurality of first in-focus images shot by a first camera of the terminal device, and determining a plurality of first lens positions corresponding to the plurality of first in-focus images respectively.”; For the rest of the 103 analysis, although imager is not explicitly mentioned in the specification, it is assumed, using BRI, that imager is synonymous to an image sensor (such as a camera), as described in the specification. Therefore, we see that a camera is being used in this paragraph to obtain a captured image); “a depth map generator to generate a depth map corresponding to the captured image” (Lu, Paragraph [0043], discloses “S302, obtaining first depth values of the interested areas of the first focused images obtained through a second camera of the terminal device, and obtaining a plurality of first depth values corresponding to the first focused images one by one. Wherein the first depth value is calculated from depth values of a plurality of different pixels of the region of interest of the first in-focus image.”; Paragraph [0051] discloses “In this embodiment, when the first camera of the terminal device captures the first target focusing image, the second camera of the terminal device is further triggered to obtain a first target depth value of the region of interest of the first target focusing image, where the plurality of first depth values include the first target depth value. For example, in a specific implementation, the second camera may be triggered to acquire the depth map of the first photographic subject when the first camera acquires the first target focus image of the first photographic subject, and a median value, an average value, a weighted average value, or other values that may reflect depth information of a region of interest in the depth map of the first photographic subject may be used as the first target depth value of the region of interest of the first target focus image.”; Here, the lens position discloses the depth map, which is ultimately determined for all the objects within the image through the obtained depth values); imager” (Lu, Paragraph [0056], discloses “When a focused image of the object of interest is acquired, the auto-focusing device may determine a lens position corresponding to the focused image, and trigger the second camera to acquire a depth value of an area of interest of the focused image.”; Here, using BRI, we can consider the lens position to be the absolute distance, because it can tell the user of the apparatus where absolutely the distance of the object in front of the second camera is).
Lu does not explicitly disclose “a map analyzer to analyze an absolute distance for a first region of the depth map based on an absolute distance of a selected sub-region of the first region of the depth map”. However, in an analogous field of endeavor, Maleki discloses “In some embodiments, the relative distance data for the object may be converted to an actual distance value (e.g., in inches, feet, meters, kilometers, etc.). To convert the relative distance data based on the depth map to an actual distance value to an object, a pre-determined relationship between the relative distance data, the focal point 113 of the first digital image 110 and the second digital image 112, the displacement factor 118 between the camera 114 and the imaginary camera 116, and/or a correction curve that compensates for an offset in distance measurements based on perceived depth in the stereoscopic image” (Maleki, Paragraph [0032]). This relates closely to analyzing an absolute distance from a depth map due to the fact that it explains how to convert relative distance data to an actual distance value using geometric factors, focal points, and correction curves. The regions here are considered to be the first digital image and the second digital image, as the first region is captured by the first digital image, and the second digital image is based on input in the form of the first digital image (which closely correlates with the sub-region). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the image analysis apparatus seen in Lu with the map analyzer seen in Maleki to achieve the same image analysis apparatus seen in Claim 1. By combining the method seen in Lu with the map analyzer seen in Maleki, one of ordinary skill in the art can effectively determine the absolute distance of the first region to get a good assessment of the objects distance away from the camera. Thus, it would have been obvious for one of ordinary skill in the art to combine the Lu and Maleki references to achieve the same image analysis apparatus seen in Claim 1.
Claim 18 recites a method with steps corresponding to the elements of the apparatus recited in Claim 1. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Lu and Maleki references, presented in rejection of Claim 1, apply to this claim.
Regarding Claim 21, Lu discloses “A non-transitory recording medium storing a computer program, which, when executed, causes at least one processor to execute a method comprising:” (Lu, Paragraph [0017], discloses: “a computer-readable storage medium, including: the computer-readable storage medium has stored therein a computer program comprising program instructions configured to, when executed by a processor, cause the processor to perform the auto-focusing method”); “controlling an imager to generate a captured image” (Lu, Paragraph [0036], discloses: “s301, in the using process of the terminal device, acquiring a plurality of first in-focus images shot by a first camera of the terminal device, and determining a plurality of first lens positions corresponding to the plurality of first in-focus images respectively.”); “generating a depth map for the captured image” (Lu, Paragraph [0043], discloses “S302, obtaining first depth values of the interested areas of the first focused images obtained through a second camera of the terminal device, and obtaining a plurality of first depth values corresponding to the first focused images one by one. Wherein the first depth value is calculated from depth values of a plurality of different pixels of the region of interest of the first in-focus image.”; Paragraph [0051] discloses “In this embodiment, when the first camera of the terminal device captures the first target focusing image, the second camera of the terminal device is further triggered to obtain a first target depth value of the region of interest of the first target focusing image, where the plurality of first depth values include the first target depth value. For example, in a specific implementation, the second camera may be triggered to acquire the depth map of the first photographic subject when the first camera acquires the first target focus image of the first photographic subject, and a median value, an average value, a weighted average value, or other values that may reflect depth information of a region of interest in the depth map of the first photographic subject may be used as the first target depth value of the region of interest of the first target focus image.”; Here, the lens position discloses the depth map, which is ultimately determined for all the objects within the image through the obtained depth values); “controlling the imager to perform an automatic focus operation on the captured image” (Lu, Paragraph [0038], discloses “the first in-focus image is an in-focus image obtained by a first camera of the terminal device by using a contrast focusing method during use of the terminal device. The sharpness of the corresponding region of interest in each first in-focus image is maximized.”; Paragraph [0049] discloses: “the image of the first shooting object is calculated until an image with the maximum definition of the interested area in the movable range of the lens is obtained;”); (Lu, Paragraph [0056], discloses “When a focused image of the object of interest is acquired, the auto-focusing device may determine a lens position corresponding to the focused image, and trigger the second camera to acquire a depth value of an area of interest of the focused image.”; Here, using BRI, we can consider the lens position to be the absolute distance, because it can tell the user of the apparatus where absolutely the distance of the object in front of the second camera is).
Lu does not explicitly disclose “determining an absolute distance based on the automatic focus operation of the imager for a selected sub-region of a first region of the depth map”. However, in a analogous field of endeavor, Maleki discloses “In some embodiments, the relative distance data for the object may be converted to an actual distance value (e.g., in inches, feet, meters, kilometers, etc.). To convert the relative distance data based on the depth map to an actual distance value to an object, a pre-determined relationship between the relative distance data, the focal point 113 of the first digital image 110 and the second digital image 112, the displacement factor 118 between the camera 114 and the imaginary camera 116, and/or a correction curve that compensates for an offset in distance measurements based on perceived depth in the stereoscopic image” (Maleki, Paragraph [0032]). As mentioned before in Claim 1, this relates with analyzing an absolute distance from a depth map due to the fact that it explains how to convert relative distance data to an actual distance value using geometric factors, focal points, and correction curves. The regions here are considered to be the first digital image and the second digital image, as the first region is captured by the first digital image, and the second digital image is based on input in the form of the first digital image (which closely correlates with the sub-region). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the image analysis recording medium seen in Lu with the technique for determining the absolute distance based on the auto-focus operation seen in Maleki to achieve the same image analysis recording medium seen in Claim 21. By combining the method seen in Lu with the technique for determining the absolute distance seen in Maleki, one of ordinary skill in the art can effectively and clearly determine the absolute distance of the first region to get a good assessment of the objects distance away from the camera. Thus, it would have been obvious for one of ordinary skill in the art to combine the Lu and Maleki references to achieve the same image analysis recording medium seen in Claim 21.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, and further in view of Choi et al. (US 2016/0198082).
Regarding Claim 3, the combination of Lu and Maleki discloses “The apparatus of claim 1, wherein the depth map comprises” (Please refer to the above-described analysis for Claim 1) . The combination of Lu and Maleki does not explicitly disclose “relative distance information of objects included in the captured image.” However, in an analogous field of endeavor, Choi discloses “relative distance information of objects included in an image” (Choi, Paragraph [0039]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine apparatus with the depth map seen in the combination of Lu and Maleki with the technique of calculating relative distance information of objects included in the captured image to allow users to comprehensively see how far each object is from the imager in each image analyzed.
Claims 4, 5, 7, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, Nobayashi et al. (JP 6800650 B2), and Satoru Komatsu (US 2015/0227815).
Regarding Claim 4, the combination of Lu and Maleki discloses “The apparatus of claim 1, wherein the at least one processor is further configured to control:” (Please refer to the above-described analysis for claim 1); (Lu, Paragraph [0058], discloses “And S303, generating a calibration relation table in which the corresponding relation between the depth values and the lens positions is stored according to the plurality of first lens positions and the corresponding plurality of first depth values.”; Here, it can be determined that the relative distances are the depth values that are computed after obtaining the first depth values and first lens positions. In other words, the ratio of relative distance and absolute distance can be determined based off of the calibration relation table between the subsequent depth values and lens positions and the first lens positions and depth values, as using this conversion table can allow somebody to see how the absolute distance can be determined based off of the relationships of the specific lens position or lens value used.)
The combination of Lu and Maleki does not explicitly disclose “an absolute distance finder to divide the depth map into a plurality of divided regions”, “determine an absolute distance” “for at least two reference regions selected from the divided regions”, “a distance converter to convert a relative distance for the first region of the depth map into an absolute distance”, or “of each of the at least two reference regions”. However, in an analogous field of endeavor, Nobayashi discloses for an absolute distance finder to divide the depth map into a plurality of divided regions that “The main subject distance dm does not necessarily have to be the subject distance corresponding to the defocus amount = 0. When the main subject occupies most of the first image signal S1, the distance at which the frequency in the frequency distribution is maximum may be set as the main subject distance dm. Further, the distance at which the frequency in the frequency distribution becomes maximum (maximum) in the vicinity of the defocus amount = 0 may be set as the main subject distance dm. The layer division process S322 divides the distance image signal into a plurality of regions (here, a region of the main subject layer and a region other than the region) as shown in FIG. 5 (E) based on the subject distance” (Nobayashi, Paragraph [0045] and Figure 5E (see below).
PNG
media_image1.png
258
348
media_image1.png
Greyscale
Figure 5E from Nobayashi
It is important to note that the main subject distance is defined as the absolute distance, as the distance for that subject is defined as the distance that has a maximum frequency distribution, which allows this to be synonymous to an absolute distance finder. Furthermore, we see in S322 the division process of the image into a plurality of regions.
Nobayashi further discloses for determining an absolute distance for at least two reference regions from the divided regions that “When the size of the vicinity region is reduced, the number of pixels set as the highly reliable subject distance is small, and the correction accuracy is low. On the other hand, if the size of the vicinity region is increased, the probability of including the distance information of different subjects as the highly reliable subject distance increases. Therefore, it is desirable that the proximity region is set so that the distance image generation unit 310 includes 1 to 8 times as many pixels as the collation region when calculating the subject distance. More preferably, it contains 3 to 5 times as many pixels.” (Nobayashi, Paragraph [0112]). As described in the aforementioned paragraph, the two reference regions are the vicinity region and proximity region. The absolute distance is considered the subject distance, as depending on the size of both the vicinity region and proximity region, it increases the reliability of the subject distance being accurate. This makes it synonymous to absolute distance. Finally, Nobayashi discloses for the first region of the depth map & the two reference regions that “The layer division process S322 divides the distance image signal into a plurality of regions (here, a region of the main subject layer and a region other than the region) as shown in FIG. 5 (E) based on the subject distance.” (Nobayashi, Paragraph [0045] and Figure 5E (see previous page)). Here, a distance image signal is interpreted as a depth map using BRI. The two reference regions as shown are the region of the main subject layer and a region other than the main region (region of the main subject layer). With that, the first region of the depth map can be interpreted as the region of the main subject layer due to it being the first one being created from the layer division process in S322. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of Lu and Maleki with the various techniques for depth map division and determining the absolute distance for at least two reference regions seen in Nobayashi to achieve the same above-described limitations for the apparatus described in Claim 4.
The combination of Lu, Maleki, and Nobayashi does not explicitly disclose “a distance converter to convert a relative distance” “into an absolute distance”. However, in an analogous field of endeavor, Komatsu discloses “a conversion table is provided for each F number so as to allow a conversion into a relative distance from the focus position on the image surface. Moreover, the obtained relative distance may be converted into an object distance using the focal length and the focus distance on an object side. The object distance may be adopted as distance information to be included in the distance map”. It is important to note that the relevant distance refers to the absolute distance, as the conversion table for each F number is used as a way to convert the estimated distance into a relevant distance that is then used to determine the actual object distance itself. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of Lu, Maleki, and Nobayashi with the distance converter seen in the Komatsu to achieve the same image analysis apparatus described in Claim 4. By combining the image analysis apparatus seen in the combination of Lu, Maleki, and Nobayashi with the distance converter seen in Komatsu, one of ordinary skill in the art can effectively understand the distance relationship between the object and the imager. Thus, it would have been obvious for one of ordinary skill in the art to combine the Lu, Maleki, Nobayashi, and Komatsu references to achieve the same image analysis apparatus seen in Claim 4
Regarding Claim 5, the combination of Lu, Maleki, Nobayashi, and Komatsu (which from here one will be described as “the combination of LMNK”) discloses “The apparatus of claim 4, wherein the at least one processor is further configured to control the absolute distance finder to select a sub-region having a relative distance within a preset range among the first region of the depth map as a reference region.” (Lu, Paragraph [0058], discloses “the preset depth value range may be a distance that the terminal device can shoot”; Nobayashi, Paragraph [0045]. discloses “The layer division process S322 divides the distance image signal into a plurality of regions (here, a region of the main subject layer and a region other than the region) as shown in FIG. 5 (E) based on the subject distance.”; Recall that the distance image signal is synonymous to the depth map. The reference regions disclosed in Nobayashi that was mentioned in claim 4 have both a subregion (region other than the region of the main subject layer) and a first region (region of the main subject layer). Therefore, if these regions were used between each other at the preset depth range disclosed in Nobayashi, the limitations of this claim would be met.)
Regarding Claim 7, the combination of LMNK discloses “The apparatus of claim 4, wherein, the at least one processor is further configured to control, based on a variation of relative distances that constitute each of the divided regions exceeding a preset threshold, the absolute distance finder to exclude a corresponding divided region from the at least two reference regions.” (Nobayashi, Paragraphs [0039]-[0041] and Figure 5B-5C, disclose the following:
PNG
media_image2.png
418
802
media_image2.png
Greyscale
PNG
media_image3.png
298
325
media_image3.png
Greyscale
PNG
media_image4.png
289
374
media_image4.png
Greyscale
As shown in the aforementioned paragraphs and figures, the variation of relative distances is clearly described in Paragraphs [0039] and [0040]. The preset threshold in this case would be the minimum value for a high signal-to-noise ratio of the image signal, because the higher the ratio, the lower the reliability. Although this set of paragraphs do not disclose "exclude a corresponding divided region from the at least two reference regions", it can be interpreted by one of ordinary skill in the art in Paragraph [0041] that the regions that have a higher signal-to-noise ratio to not be as reliable for image analysis of the distance between the object compared to the regions of higher reliability. This would ultimately mean that those unreliable regions would not be considered when assessing the distances of different objects from the camera depending on the region.)
Claim 19 recites a method with steps corresponding to the elements of the apparatus recited in Claim 4. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Lu, Maleki, Nobayashi, and Komatsu references, presented in rejection of Claim 4, apply to this claim.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, Nobayashi, and Komatsu, and further in view of Celik et al. (US 2019/0373162) and Tomita et al. (CN 103210332 B).
Regarding Claim 6, the combination of LMNK discloses “The apparatus of claim 4, wherein the at least one processor is further configured to control the absolute distance finder to” (Please refer to the above-described analysis for Claim 4); (Nobayashi, Paragraph [0045], discloses “…The layer division process S322 divides the distance image signal into a plurality of regions (here, a region of the main subject layer and a region other than the region)…”); at least one region” or “having a minimum relative distance or a maximum relative distance” “from the selection of the reference region”. However, Celik discloses for excluding at least one region that “For background and foreground objects, Boolean mask operations may be applied to define the extent of each object and to preferentially include or exclude these regions within regions of interest” (Celik, Paragraph [0077]). As shown, at least one region from the regions of interest is being excluded within the number of regions of interest. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of LMNK with the Celik technique of excluding at least one region to achieve the above-described limitations for the image analysis apparatus.
The combination of Lu, Maleki, Nobayashi, Komatsu, and Celik does not explicitly disclose “having a minimum relative distance or a maximum relative distance” “from the selection of the reference region”. However, in an analogous field of endeavor, Tomita discloses “in step S205, distance information (reference distance Ds) that is a candidate for the reference region among the focus search region candidates is acquired” (Tomita, Paragraph [0131]). Tomita further discloses “Only the distance information within a certain range from the reference distance Ds is retained as the focus detection area candidate, and the distance information exceeding a certain distance from the reference distance is excluded from the focus detection area candidates as the selection target (step S206). Then, the focus detection area candidate having the smallest distance information is selected from among the focus detection area candidates reserved as selection targets (step S207)” (Tomita, Paragraph [0132]). We can see from here that at least one region here has a minimum distance, which is further used as a focus detection area candidate. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of Lu, Maleki, Nobayashi, Komatsu, and Celik with the Tomita technique of selecting one region having a minimum relative distance to achieve a more complete image analysis apparatus. By combining the Tomita technique with the apparatus seen in the combination of Lu, Maleki, Nobayashi, Komatsu, and Celik, one of ordinary skill in the art allows for a user to determine the minimum distance observed for a region in order to focus on the key regions that have the distances that are the closest to the imager. Therefore, it would have been obvious for one of ordinary skill in the art to combine the Lu, Maleki, Nobayashi, Komatsu, Celik and Tomita references to achieve the same apparatus described in Claim 6.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, Nobayashi, and Komatsu, and further in view of Lee et al. (US 2016/0042526).
Regarding Claim 8, the combination of LMNK discloses “The apparatus of claim 4, wherein the at least one processor is further configured to control the absolute distance finder to” (Please refer to the above-described analysis regarding Claim 4); (Lee, Paragraph [0119]). Here, subregions can be considered reference regions in this case, as they are being used to determine which area within the image has the shortest distance away from the camera to be able to determine a focal region. Lee further discloses “In the case of the group having the largest number of grouped sub-regions of interest, the processor 120 may determine the focal region through grasping of the number of sub-regions of interest of each group, but is not limited thereto. It is also possible to select the group having the specific number of grouped sub-regions of interest. Further, if there are several groups having the same number of sub-regions of interest, the processor 120 may determine the focal region using one of the group having the shortest distance to the object and the group having the highest reliability among the several groups.” (Lee, Paragraph [0122]). Lee finally discloses “The determining the focal region (operation S1330) may include determining one of a group having the shortest distance to the object, a group having the highest reliability, and a group having the largest number of grouped sub-regions of interest as the focal region” (Lee, Paragraph [0185]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of LMNK with the technique of determining a number of reference regions seen in Lee to have an improved image analysis apparatus. By incorporating the Lee technique of determining a number of reference regions with the apparatus seen in the combination of LMNK, one of ordinary skill in the art can assess which region can be used to determine the distances between the object and the camera. Therefore, it would have been obvious to combine the Lu, Maleki, Nobayashi, Komatsu, and Lee references to achieve the same apparatus described in Claim 8.
Regarding Claim 9, the combination of Lu, Maleki, Nobayashi, Komatsu, and Lee discloses “The apparatus of claim 8, wherein the at least one processor is further configured to control the absolute distance finder to increase the number of reference regions based on an increase in the variation of the relative distances constituting the depth map.” (Lee, Paragraph [0122], discloses “In the case of the group having the largest number of grouped sub-regions of interest, the processor 120 may determine the focal region through grasping of the number of sub-regions of interest of each group, but is not limited thereto. It is also possible to select the group having the specific number of grouped sub-regions of interest. Further, if there are several groups having the same number of sub-regions of interest, the processor 120 may determine the focal region using one of the group having the shortest distance to the object and the group having the highest reliability among the several groups”; In [0122], we can see here that the number of reference regions can be increased depending on what has been selected, and each group has subregions that have different distances that each equate to allowing the group to have a shorter or longer distance away from the object.)
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, Nobayashi, and Komatsu, and further in view of Celik.
Regarding Claim 14, the combination of LMNK discloses “The apparatus of claim 4, wherein the at least one processor is further configured to control the absolute distance finder to: perform an automatic focus operation on the capturing region of the captured image corresponding to the reference region” (Lu, Paragraph [0038], discloses “the first in-focus image is an in-focus image obtained by a first camera of the terminal device by using a contrast focusing method during use of the terminal device. The sharpness of the corresponding region of interest in each first in-focus image is maximized”); imager corresponding to a maximum sharpness of an auto focus image from the performed automatic focus operation”. However, Celik discloses “As shown in the plot of FIG. 5A, the camera distance 502 is the distance that correlates to the highest image sharpness scalar value 504 (e.g., approximate optimal focus distance for the depth level). Lesser image sharpness scalar values 506a and 506b are shown on each side of the highest image sharpness scalar value 504 at corresponding distances that are less and more than the distance 502 that produces the highest image sharpness scalar value 504. These distances are separated by the incremental value used by the motor 116 to step the camera and lens assembly 102 up to and then past the distance 502 that produces the highest image sharpness scalar value 504 (e.g., the approximate optimal focus distance of the depth level)” (Celik, Paragraph [0070] and Figure 5A (see below)).
PNG
media_image5.png
336
476
media_image5.png
Greyscale
Here, the optimal focus distance is synonymous to the maximum distance, as the optimal distance is being determined by the lens position (lens being moved by the motor) and the distances that correlate to the highest image sharpness scalar value. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of invention to combine the apparatus seen in the combination of LMNK with the Celik technique of determining the absolute distance based on the maximum sharpness of an autofocus image to result in an improved image analysis apparatus. By doing this, one of ordinary skill in the art would enable a more accurate distance calculation within the image due to the sharper and clearer image that is selected to find the absolute distance. Thus, it would have been obvious for one of ordinary skill in the art to combine the Lu, Maleki, Nobayashi, Komatsu, and Celik references to achieve the same apparatus described in Claim 14.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, Nobayashi, and Komatsu, and further in view of Furukawa et al. (US 2017/0256062) and Celik.
Regarding Claim 15, the combination of LMNK discloses “The apparatus of claim 4, wherein the at least one processor is further configured to control the absolute distance finder to: perform an automatic focus operation on” (Lu, Paragraph [0038], discloses “the first in-focus image is an in-focus image obtained by a first camera of the terminal device by using a contrast focusing method during use of the terminal device. The sharpness of the corresponding region of interest in each first in-focus image is maximized.”) to a maximum sharpness of an auto focus image from the performed automatic focus operation”. However, in an analogous field of endeavor, Furukawa discloses “the reference dimension information contains an amount defined by the dimension of a region preset as a region of interest as a reference (a reference region of interest), and by the range of the boundary between this region and its peripheral region.” (Furukawa, Paragraph [0027]). As we can see here, the reference region, peripheral region, and enlarged region are disclosed in these two paragraphs. Furukawa further discloses that “the generator 106 generates an image by enlarging or reducing the image size of a rough extraction image in accordance with the scaling coefficient, and acquires dimension information for a rough extraction region drawn in the generated image.” (Furukawa, Paragraph [0064]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in Lu, Maleki, Nobayashi, and Komatsu with the Furukawa technique of acquiring an image of an enlarged capturing region based on peripheral region of the reference region to have an improved image analysis apparatus, By combining the apparatus seen in Lu, Maleki, Nobayashi, and Komatsu with the Furukawa technique of having an image with an enlarged capturing region, one of ordinary skill in the art enables a user to focus on a particular object to gain a clearer representation of the object in order to determine the distance accurately using the apparatus. Thus, it would have been obvious for one of ordinary skill in the art to combine the Lu, Maleki, Nobayashi, Komatsu, and Furukawa references to achieve the above-described limitations in Claim 15.
The combination of Lu, Maleki, Nobayashi, Komatsu, and Furukawa does not explicitly disclose “determine the absolute distance of the reference region based on a position of a focus lens of the imager corresponding to a maximum sharpness of an auto focus image from the performed automatic focus operation”. However, in an analogous field of endeavor, Celik discloses “As shown in the plot of FIG. 5A, the camera distance 502 is the distance that correlates to the highest image sharpness scalar value 504 (e.g., approximate optimal focus distance for the depth level). Lesser image sharpness scalar values 506a and 506b are shown on each side of the highest image sharpness scalar value 504 at corresponding distances that are less and more than the distance 502 that produces the highest image sharpness scalar value 504. These distances are separated by the incremental value used by the motor 116 to step the camera and lens assembly 102 up to and then past the distance 502 that produces the highest image sharpness scalar value 504 (e.g., the approximate optimal focus distance of the depth level).” (Celik, Paragraph [0070]). Here, the optimal focus distance is synonymous with the maximum or absolute distance, as the optimal distance is being determined by the lens position (lens being moved by the motor) and the distances that correlate to the highest image sharpness scalar value. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of invention to combine the apparatus seen in the combination of Lu, Maleki, Nobayashi, Komatsu, and Furukawa with the Celik technique of determining the absolute distance based on the maximum sharpness of an autofocus image to result in an improved image analysis apparatus. By doing this, one of ordinary skill in the art would enable a more accurate distance calculation within the image due to the sharper and clearer image that is selected to find the absolute distance. Therefore, it would have been obvious for one of ordinary skill in the art to combine the Lu, Maleki, Nobayashi, Komatsu, Furukawa, and Celik references to achieve the same apparatus described in Claim 15.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, and further in view of Shimauchi et al. (US 2019/0158732).
Regarding Claim 17, the combination of Lu and Maleki discloses “The apparatus of claim 1, wherein the at least one processor is further configured to” (Please refer to the above described analysis regarding Claim 17) Shimauchi, Paragraph [0098]). Using BRI, "resolution map" is being interpreted as the resolution of the depth map. From Paragraph [0098], we can see how the depth map resolution is being converted based on a defined threshold into a resolution map, and as a result, the image can be modified based off of the acquired resolution map later on (as seen in [0117] with the imaging element and image processing unit coincidingly working together to adjust the resolution of the image based on the resolution map). Shimauchi further discloses that “In the imaging element 22 and the image processing unit 23 illustrated in FIG. 2 or 3, in a case in which the resolution control unit 107 controls a resolution on the basis of a resolution map, the resolution control unit 107 gives an instruction to, for example, the read-out control unit 102 and controls the number of pixels to be read out from the pixel array portion 101 to control a resolution.” (Shimauchi, Paragraph [0117] and Figures 2-3 (see below)).
PNG
media_image6.png
421
619
media_image6.png
Greyscale
PNG
media_image7.png
424
624
media_image7.png
Greyscale
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of invention to combine the apparatus seen in the combination of Lu, Maleki, Nobayashi, Komatsu, and Furukawa with the Celik technique of determining the absolute distance based on the maximum sharpness of an autofocus image to result in an improved image analysis apparatus. By doing this, one of ordinary skill in the art would enable a more accurate distance calculation within the image due to the sharper and clearer image that is selected to find the absolute distance. Thus, it would have been obvious for one of ordinary skill in the art to combine the Lu, Maleki, Nobayashi, Komatsu, Furukawa, and Celik references to achieve the same apparatus described in Claim 15.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Lu in view of Maleki, and further in view of Nobayashi.
Regarding Claim 20, the combination of Lu and Maleki discloses “The method of claim 18, wherein the determining the absolute distance comprises selecting a region having a relative distance included within a preset range” (Lu, Paragraph [0058], discloses “the preset depth value range may be a distance that the terminal device can shoot”); (Nobayashi, Paragraph [0045]). Remember from the analysis of Claim 4 that the reference regions disclosed in Nobayashi have both a subregion (region other than the region of the main subject layer) and a first region (region of the main subject layer). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method seen in the combination of Lu and Maleki with the technique of having the first region of the depth map as a reference region seem in Nobayashi to allow for easy selection between different objects bounded by different regions. By combining the method seen in the combination of Lu and Maleki with the technique of having a first region of the depth map as a reference region, one of ordinary skill in the art allows a user to easily select an area of interest where a moving or stationary object is within the view of the imager to observe the absolute distance that it has from the imager. Therefore, it would have been obvious to combine the Lu, Maleki, and Nobayashi references to achieve the same method described in Claim 20.
Allowable Subject Matter
Claims 10-13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: For Claim 10, the combination of LMNK does not explicitly disclose generating a depth map array based on a size of a plurality of relative distances corresponding to the divided regions, respectively, dividing the depth map array into a plurality of depth map groups according to the size of the relative distance, selecting a reference map group as a selection reference for the reference region among the plurality of depth map groups, extracting a relative distance from the reference map group, and selecting a divided region corresponding to the relative distance extracted from the reference map group as the reference region. None of the cited prior art references provide a motivation to teach the ordered combination with the claim limitations of Claim 10. Claims 11-13 includes the above-described allowable subject matter due to their dependency from Claim 10, either directly or indirectly.
Claim 16 is objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: For Claim 16, while the combination of LMNK does not explicitly disclose performing a first automatic focus operation on an enlarged capturing region corresponding to the reference region and a peripheral region of the reference region of the captured image, performing a second automatic focus operation on a reference focus region corresponding to the reference region of an auto focus image from the performed first automatic focus operation, and determining an absolute distance of the reference region based on a position of a focus lens of the imager corresponding to a maximum sharpness of a first auto focus image from the performed first automatic focus operation and the sharpness of a second auto focus image from the performed second automatic focus operation. None of the cited prior art references provide a motivation to teach the ordered combination with the claim limitations of Claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Laganakos et al. (US 2021/0168348) teaches a method comprising the steps of obtaining image data captured at and using a focus configuration.
Safee-Rad et al. (US 2009/0167930) teaches A method and apparatus improves an auto focus system by altering, such as by positioning, at least one lens of a digital camera to a plurality of predetermined nonuniform lens positions corresponding to predetermined nonuniform lens position data.
Wong et al. (US 2011/0142287) teaches a two picture matching curve information that is able to be used to determine precise object distance or relative object distance in a scene.
Yoon et al. (US 2015/0187083) teaches an electronic apparatus that processes the image information based on the distance information.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SORIE I KOROMA JR whose telephone number is (571)272-9259. The examiner can normally be reached Monday - Friday 8AM-6:00PM; Alternate Fridays Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SORIE I KOROMA JR/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights