Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 and 11-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process using collected image data based on the visualization of collected image data and evaluating and judging the object area in the image data and plurality of locations inside the area (concept performed in a human mind, including as observation, evaluation, judgment, prediction, etc.). This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved. The claims 1-9 and 11-20 do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally based on evaluation, prediction, judgement and no additional features in the claims would preclude them from being performed as such except for the generic computer elements and generic display recited as high level of generality (i.e., processor and generic display)
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1 and 19 are directed to an abstract idea as shown below:
Regarding independent claims 1 and 19.
STEP 1: Do the claims fall within one of the statutory categories?
YES.
Claims 1 and 19 are directed to an information processing apparatus and an information processing method which satisfy system and process requirement.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
YES.
The claims are directed toward a mental process (i.e. abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
Claims 1 and 19 comprise a mental process that can be practicably performed in the human mind based on observation, evaluation, judgment, prediction, etc., based on the visualizing the image data (except for generic optical sensor, generic computers or components and generic display) and, therefore, an abstract idea.
Regarding claim(s) 1 and 19: (representative claim 1)
An information processing apparatus comprising:
a memory storing instructions; and a processor configured to execute the instructions to (generic computing elements i.e., processor, memory and instructions):
determine an object area that includes an object in a first image obtained by imaging of the object through an optical system and an image sensor (collection of imaging data using generic imaging/optical sensor which is insignificant extra solution activity and mental process person intelligence of observation, evaluation, judgment, prediction of detecting object area in the collected image data);
acquire, at a plurality of locations in the object area, first information that varies according to parallelism of the image sensor to the object (mental process i.e., person intelligence of observation, evaluation, judgment, prediction of detecting plurality of locations in the object area and detecting information that varies according to parallelism of the image sensor to the object based on mental process of evaluation and judgment);
generate second information as a guide for a user adjusting the parallelism according to the first information; and output a second image including an image of the object area and the second information (mental process of evaluation and judgment i.e. person intelligence and person/human activity of guiding the user for adjusting the parallelism and display the second image which is insignificant post solution activity).
The above limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind by a person/human intelligence . Furthermore limitations, “ a memory storing instructions; and a processor configured to execute the instructions to (generic computing elements i.e., processor, memory and instructions): determine an object area that includes an object in a first image obtained by imaging of the object through an optical system and an image sensor (collection of imaging data using generic imaging/optical sensor which is insignificant extra solution activity and mental process person intelligence of observation, evaluation, judgment, prediction of detecting object area in the collected image data), acquire, at a plurality of locations in the object area, first information that varies according to parallelism of the image sensor to the object (mental process i.e., person intelligence of observation, evaluation, judgment, prediction of detecting plurality of locations in the object area and detecting information that varies according to parallelism of the image sensor to the object based on mental process of evaluation and judgment), generate second information as a guide for a user adjusting the parallelism according to the first information; and output a second image including an image of the object area and the second information (mental process of evaluation and judgment i.e. person intelligence and person/human activity of guiding the user for adjusting the parallelism and display the second image which is insignificant post solution activity)” are insignificant.
The Examiner notes that under MPEP 2106.04(A) (2) (III), the courts consider a mental process (thinking, human intelligence) that can be performed in the mind/intelligence using a paper and pencil to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[Mental processes and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978).
Other than generic and conventional computing hardware/instructions, generic imaging/optical sensing elements and generic/conventional display recited in the independent claims 1 and 19 and disclosed in the specification, nothing in the independent claims elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process based on the person/human intelligence and mental process. The generic and conventional computing elements, imaging/optical sensing element and generic display recited in independent claims 1 and 19 are a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing elements, generic imaging/optical elements and generic display are recited as just to automate the mental process. Furthermore the display is insignificant post solution activity without any detail (Step 2A, prong 1 Test Abstract idea = Yes).
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim(s) 1 and 19 do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. Claim(s) 1 and 19 recite(s) the further limitations of: “a memory storing instructions; and a processor configured to execute the instructions to (generic computing elements i.e., processor, memory and instructions): determine an object area that includes an object in a first image obtained by imaging of the object through an optical system and an image sensor (collection of imaging data using generic imaging/optical sensor which is insignificant extra solution activity and mental process person intelligence of observation, evaluation, judgment, prediction of detecting object area in the collected image data), acquire, at a plurality of locations in the object area, first information that varies according to parallelism of the image sensor to the object (mental process i.e., person intelligence of observation, evaluation, judgment, prediction of detecting plurality of locations in the object area and detecting information that varies according to parallelism of the image sensor to the object based on mental process of evaluation and judgment), generate second information as a guide for a user adjusting the parallelism according to the first information; and output a second image including an image of the object area and the second information (mental process of evaluation and judgment i.e. person intelligence and person/human activity of guiding the user for adjusting the parallelism and display the second image which is insignificant post solution activity)”.
The above limitations are recited at a high level of generality (i.e. as a general action of mental process based on acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity without further detail. Furthermore, the claims are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Other than generic and conventional computing hardware/instructions, generic imaging/optical sensing elements and generic/conventional display recited in the independent claims 1 and 19 and disclosed in the specification, nothing in the independent claims elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process based on the person/human intelligence and mental process. The generic and conventional computing elements, imaging/optical sensing element and generic display recited in independent claims 1 and 19 are a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing elements, generic imaging/optical elements and generic display are recited as just to automate the mental process. Furthermore the display is insignificant post solution activity without any detail (Step 2A, prong 2 Test Abstract idea = Yes).
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
NO.
The claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Other than generic and conventional computing hardware/instructions, generic imaging/optical sensing elements and generic/conventional display recited in the independent claims 1 and 19 and disclosed in the specification, nothing in the independent claims, elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process based on the person/human intelligence and mental process. The generic and conventional computing elements, imaging/optical sensing element and generic display recited in independent claims 1 and 19 are a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing elements, generic imaging/optical elements and generic display are recited as just to automate the mental process. Furthermore the display is insignificant post solution activity without any detail
Thus, since Claim(s) 1 and 19 are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that Claim(s) 1 and 19 are not eligible subject matter under 35 U.S.C 101 (Step 2B, Test Abstract idea = Yes).
Regarding dependent claims 2-9, 11-18 and 20, claims 2-9, 11-18 and 20 further limit the abstract idea of performance of the limitations in the mind based on mental process of observations, judgement, evaluation, and thought process. Other than generic and conventional computing elements/hardware/instructions, generic/conventional imaging/optical elements and generic and conventional display elements recited in the independent claims 2-9, 11-18 and 20 and disclosed in the specification, nothing in the independent/dependent claim (s) elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using based on the human intelligence and mental process. The generic and conventional computing elements, generic/conventional imaging/optical element and generic display element recited in independent claims is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). Independent/dependent claims recite generic and conventional computing elements hardware/instructions, generic/conventional imaging/optical elements and generic and conventional display elements are recited as just to automate the mental process. Furthermore the display control unit is insignificant post solution activity without any detail. There are no additional elements in the claims that would integrate the abstract idea into a practical application. The claims do not mention any improvement to a computer or to any other technology or technical field. The limitations of claims 2-9, 11-18 and 20 fail to add inventive concept to otherwise mental process. Therefore the dependent claims 2-9, 11-18 and 20 are no more than abstract idea without significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over ASUKABE et al. (US 20220191401) .
Regarding claims 1 and 19 ASUKABE disclose information processing apparatus/method (ASUKABE Figs. 3, 5-6 and 18, paragraphs 0096-0099, 0103-0106 and 0152-0154 information processing apparatus/method) comprising:
a memory storing instructions; and a processor configured to execute the instructions to (AUKABE Figs 3-5, paragraphs 0152-0153 and 0418-0421, discloses a memory storing instructions; and a processor configured to execute the instructions ):
determine an object area that includes an object in a first image obtained by imaging of the object through an optical system and an image sensor (ASUKABE Figs. 3 and 18, paragraph 0181 ASUKABE states the captured image content is, for example, a setting mode in the imaging device 1. For example, when a face detection mode is set, the target region setting unit 33 detects a face region through image analysis processing and sets the detected face region as the target region. When a pupil detection mode is set, the target region setting unit 33 detects a pupil region through image analysis processing and sets the detected pupil region as a target region and paragraph 0182 ASUKABE states the target region setting unit 33 can set, for example, a region in a captured image designated through a user operation as the target region. This obviously corresponds to determine an object area that includes an object in a first image obtained by imaging of the object through an optical system and an image sensor);
acquire, at a plurality of locations in the object area, first information that varies according to parallelism of the image sensor to the object (ASUKABE Fig. 18, paragraph 0017 discloses the image processing device according to the present technology further includes a target region setting unit configured to set a target region in accordance with captured image content, and the map data generation unit generates the defocus map data at a plurality of positions in the target region. Thus, data of each of the defocus amounts at the plurality of positions in the target region is calculated and paragraph 0077 states Defocus map data indicates a defocus amount of each position in a captured image or a target region in the captured image. A defocus amount quantitively indicates a defocus (blurring) state at a certain position in a captured image and corresponds to, for example, a diameter of a defocus circle. Based on the amounts of defocus position of regions in image obviously provide the information that varies according to the parallelism of the image sensor i.e. to object and ASUKABE paragraph 0116 disclose On the imaging surface of the imaging element 100B, an imaging pixel group 105 formed by imaging pixels of 2 columns×2 rows for imaging an image of a subject and a pair of image surface phase difference pixels 106 for detecting a phase difference of optical images of the subject are provided. The one pair of image surface phase difference pixels 106 are separately disposed between the plurality of imaging pixels on the imaging surface. In the system of ASUKABE phase differences of image pixels group and amount of defocus in image area would obviously provide information that varies according to parallelism of the image sensor to the object i.e., target object);
generate second information as a guide for a user adjusting the parallelism according to the first information; and output a second image including an image of the object area and the second information (ASUKABE Fig. 18, paragraphs 0165-0166 disclose The map data generation unit 31 generates defocus map data that is calculated from a phase difference signal (phase difference information) detected by the image surface phase difference pixels in the imaging element unit 12 and indicate defocus amounts at a plurality of positions of a captured image by the imaging element unit 12. For example, when the X axis represents the horizontal direction of a captured image and the Y axis represents the vertical direction in FIG. 7, the map data generation unit 31 generates values of defocus amounts (DF1, DF2, DF3, . . . ) at positions specified by the X axis coordinates (X1, X2, X3, . . . ) and Y axis coordinates (Y1, Y2, Y3, . . . ) in the captured image as the defocus map data and ASUKABE paragraph 0380 disclose According to an embodiment, the image processing device 30 mounted on the imaging device 1 includes: the map data generation unit 31 configured to generate defocus map data which is calculated from phase difference information detected by image surface phase difference pixels in the imaging element unit 12 and indicates defocus amounts at a plurality of positions of a captured image by an imaging element unit 12; and the operation control unit 34 configured to perform imaging operation control using the defocus map data generated by the map data generation unit 31 [see FIG. 24 and Fig. 18], paragraph 0346 disclose superimposition display of the map image on the captured image is useful for a person who performs focusing to perform the imaging with the imaging device 1 to adjust the focus or diaphragm mechanism and paragraph 0388 disclose the image processing device 30 performs operation control of the focus lens or the diaphragm mechanism in accordance with the detected part. The image processing device 30 acquires the fixed-value information of the defocus amount at each position of the target region associated with each part of the pupil region, for example, the eyelash region, and performs operation control of the diaphragm mechanism such that the fixed value of the defocus amount at each position set in the target region is obtained. The image processing device 30 may perform focus lens control of the lens system 11. All this in the system ASUKABE corresponds to generate second information as a guide for a user adjusting the parallelism according to the first information; and output a second image including an image of the object area and the second information).
:
Therefore it would be obvious before the filing data of the claimed invention to determine an object area that includes an object in a first image obtained by imaging of the object through an optical system, acquire at a plurality of locations in the object area, first information that varies according to parallelism of the image sensor to the object; generating second information as a guide for a user adjusting the parallelism according to the first information; and output a second image including an image of the object area and the second information as shown by ASUKABE because such a system/process provide automated system to control and adjust of the defocus and blurring areas in the captured image.
Regarding claim 2 ASUKABE disclose processor disposes the second information at a position according to each of the plurality of locations in the second image (ASUKABE Figs. 18 and 24, paragraphs 0165-0166 disclose The map data generation unit 31 generates defocus map data that is calculated from a phase difference signal (phase difference information) detected by the image surface phase difference pixels in the imaging element unit 12 and indicate defocus amounts at a plurality of positions of a captured image by the imaging element unit 12. For example, when the X axis represents the horizontal direction of a captured image and the Y axis represents the vertical direction in FIG. 7, the map data generation unit 31 generates values of defocus amounts (DF1, DF2, DF3, . . . ) at positions specified by the X axis coordinates (X1, X2, X3, . . . ) and Y axis coordinates (Y1, Y2, Y3, . . . ) in the captured image as the defocus map data and ASUKABE paragraph 0380 disclose According to an embodiment, the image processing device 30 mounted on the imaging device 1 includes: the map data generation unit 31 configured to generate defocus map data which is calculated from phase difference information detected by image surface phase difference pixels in the imaging element unit 12 and indicates defocus amounts at a plurality of positions of a captured image by an imaging element unit 12; and the operation control unit 34 configured to perform imaging operation control using the defocus map data generated by the map data generation unit 31 [see FIG. 24 and Fig. 18).
Regarding claim 3 ASUKABE disclose processor determines the object area using information regarding a distance from the image sensor obtained at a plurality of positions in the first image (ASUKABE paragraph 0167-0169 disclose The map data generation unit 31 can calculate subject distances at a plurality of positions of a captured image by the imaging element unit 12 based on the generated defocus map data and lens information and generate depth map data indicating the calculated subject distances, For example, in FIG. 8, the map data generation unit 31 generates values of the subject distances (DP1, DP2, DP3, . . . ) at positions specified by the X axis coordinates (X1, X2, X3, . . . ) and Y axis coordinates (Y1, Y2, Y3, . . . ) in the captured image as the depth map data and the display control unit 32 generates a defocus map image indicating a distribution of defocus amounts in the captured image using the defocus map data generated by the map data generation unit 31 and performs display control. This obviously corresponds to processor determines the object area using information regarding a distance from the image sensor obtained at a plurality of positions in the first image).
Regarding claim 4 ASUKABE disclose processor determines the object area using information regarding a distance from the image sensor obtained at a plurality of positions in the first image (ASUKABE paragraph 0263 discloses The selected region selected through the user operation may be a selected region set in advance by the user. For example, a predetermine defocus amount or subject distance is set and the image processing device 30 can also set a region corresponding to the predetermined defocus amount or subject distance as the target region in step S205. In this case, information generated in a previous frame may be used as the defocus amount or the subject distance in the captured image. This obviously processor determines the object area using information regarding a distance from the image sensor obtained at a plurality of positions in the first image).
Regarding claim 5 ASUKABE disclose a plurality of pixels on the image sensor include a plurality of photoelectric conversion elements, respectively, the photoelectric conversion elements being configured to receive light from areas different from each other in a pupil of the optical system (ASUKABE 0103-0104 disclose the imaging element unit 12 includes, for example, a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) type image sensor 12a (an imaging element). The image sensor 12a is configured by imaging pixels for imaging an image of a subject and image surface phase difference pixels for detecting a phase difference of an optical image of the subject and The imaging element unit 12 performs, for example, correlated double sampling (CDS) processing and automatic gain control (AGC) processing, as well as analog/digital (A/D) conversion processing on an electrical signal obtained by photoelectrically converting light received by the image sensor 12a. The imaging element unit 12 outputs an imaging signal as digital data to the camera signal processing unit 13 or the camera control unit 18. Therefore it is obvious in system of ASUKABE a plurality of pixels on the image sensor include a plurality of photoelectric conversion elements, respectively, the photoelectric conversion elements being configured to receive light from areas different from each other in a pupil of the optical system ), and
the processor acquires the first information using a phase difference between a pair of image signals generated by using signals output from the plurality of photoelectric conversion elements of the plurality of pixels (ASUKABE paragraph 0116 On the imaging surface of the imaging element 100B, an imaging pixel group 105 formed by imaging pixels of 2 columns×2 rows for imaging an image of a subject and a pair of image surface phase difference pixels 106 for detecting a phase difference of optical images of the subject are provided. The one pair of image surface phase difference pixels 106 are separately disposed between the plurality of imaging pixels on the imaging surface and ASUKABE paragraph 0120 disclose as described above, the image surface phase difference pixels are integrated with the R, G, and B imaging pixels or are disposed in the periphery. Therefore, a defocus amount can be accurately calculated in pixel units of several pm from the read phase difference signal).
Regarding claim 6 ASUKABE disclose the second information represents a defocus direction of the optical system relative to the object and an in-focus state of the optical system on the object (ASUKABE paragraph 0166 disclose For example, when the X axis represents the horizontal direction of a captured image and the Y axis represents the vertical direction in FIG. 7, the map data generation unit 31 generates values of defocus amounts (DF1, DF2, DF3, . . . ) at positions specified by the X axis coordinates (X1, X2, X3, . . . ) and Y axis coordinates (Y1, Y2, Y3, . . . ) in the captured image as the defocus map data. In the system of ASUKABE coordinates would obviously provide the direction of defocus).
Regarding claim 7 ASUKABE disclose the second information represents distances of the image sensor from positions parallel to the object at the plurality of locations (ASUKABE paragraph 0167-0168 disclose map data generation unit 31 can calculate subject distances at a plurality of positions of a captured image by the imaging element unit 12 based on the generated defocus map data and lens information and generate depth map data indicating the calculated subject distances and For example, in FIG. 8, the map data generation unit 31 generates values of the subject distances (DP1, DP2, DP3, . . . ) at positions specified by the X axis coordinates (X1, X2, X3, . . . ) and Y axis coordinates (Y1, Y2, Y3, . . . ) in the captured image as the depth map data. This obviously corresponds to the second information represents distances of the image sensor from positions parallel to the object at the plurality of locations ).
Regarding claim 8 ASUKABE disclose the processor outputs the second image to a display unit configured to display an image (ASUKABE paragraph 0380 disclose According to an embodiment, the image processing device 30 mounted on the imaging device 1 includes: the map data generation unit 31 configured to generate defocus map data which is calculated from phase difference information detected by image surface phase difference pixels in the imaging element unit 12 and indicates defocus amounts at a plurality of positions of a captured image by an imaging element unit 12; and the operation control unit 34 configured to perform imaging operation control using the defocus map data generated by the map data generation unit 31see FIG. 24 and Fig. 18] and paragraph 0346 disclose superimposition display of the map image on the captured image is useful for a person who performs focusing to perform the imaging with the imaging device 1 to adjust the focus or diaphragm mechanism and paragraph 0388 disclose the image processing device 30 performs operation control of the focus lens or the diaphragm mechanism in accordance with the detected part. The image processing device 30 acquires the fixed-value information of the defocus amount at each position of the target region associated with each part of the pupil region, for example, the eyelash region, and performs operation control of the diaphragm mechanism such that the fixed value of the defocus amount at each position set in the target region is obtained. The image processing device 30 may perform focus lens control of the lens system 11.This corresponds to the processor outputs the second image to a display unit configured to display an image).
Regarding claim 9 ASUKABE disclose the processor outputs the second image to a display unit configured to display an image (ASUKABE Fig. 10 paragraph 0193 disclose the defocus map image 61 is superimposed and displayed on the captured image 60 and is displayed like a heatmap, for example, by performing coloring in accordance with a defocus amount of each position of the captured image. Thus, the defocus amount of each position in the whole captured image can be visually recognized).
Regarding claim 10 ASUKABE disclose determines a position at which the second information is disposed by using a length between a pixel in the object area and any of pixels at four corners of the first image (Fig 4, paragraph 0109 disclose In FIG. 4, each part of the pixel arrays of imaging elements 100A and 100B is illustrated as an example of the pixel array of the image sensor 12a (the imaging element) , paragraph 0111 also disclose pixel positions, paragraph 0165 disclose The map data generation unit 31 generates defocus map data that is calculated from a phase difference signal (phase difference information) detected by the image surface phase difference pixels in the imaging element unit 12 and indicate defocus amounts at a plurality of positions of a captured image by the imaging element unit 12, paragraph 0380 disclose According to an embodiment, the image processing device 30 mounted on the imaging device 1 includes: the map data generation unit 31 configured to generate defocus map data which is calculated from phase difference information detected by image surface phase difference pixels in the imaging element unit 12 and indicates defocus amounts at a plurality of positions of a captured image. This corresponds to determines a position at which the second information (map ) is disposed by using a pixel in the object area and any of pixels at four corners of the first image. However ASUKABE has not explicitly disclose determining length between pixel however it would be obvious to determine the length between pixel based on the position of pixels.
Regarding claim 11 ASUKABE disclose 11 disclose the processor changes the object area according to a user operation (ASUKABE Fig. 18, paragraph 0231The user can view the size of the circular icon BC displayed as the defocus map image and recognize the defocus amount of each position in the captured image 60 visually and sensually. paragraphs 0165-0166 disclose the map data generation unit 31 generates defocus map data that is calculated from a phase difference signal (phase difference information) detected by the image surface phase difference pixels in the imaging element unit 12 and indicate defocus amounts at a plurality of positions of a captured image by the imaging element unit 12. For example, when the X axis represents the horizontal direction of a captured image and the Y axis represents the vertical direction in FIG. 7, the map data generation unit 31 generates values of defocus amounts (DF1, DF2, DF3, . . . ) at positions specified by the X axis coordinates (X1, X2, X3, . . . ) and Y axis coordinates (Y1, Y2, Y3, . . . ) in the captured image as the defocus map data paragraph 0380 disclose According to an embodiment, the image processing device 30 mounted on the imaging device 1 includes: the map data generation unit 31 configured to generate defocus map data which is calculated from phase difference information detected by image surface phase difference pixels in the imaging element unit 12 and indicates defocus amounts at a plurality of positions of a captured image by an imaging element unit 12; and the operation control unit 34 configured to perform imaging operation control using the defocus map data generated by the map data generation unit 31 [see FIG. 24 and Fig. 18], paragraph 0346 disclose superimposition display of the map image on the captured image is useful for a person who performs focusing to perform the imaging with the imaging device 1 to adjust the focus or diaphragm mechanism and paragraph 0388 disclose the image processing device 30 performs operation control of the focus lens or the diaphragm mechanism in accordance with the detected part. The image processing device 30 acquires the fixed-value information of the defocus amount at each position of the target region associated with each part of the pupil region, for example, the eyelash region, and performs operation control of the diaphragm mechanism such that the fixed value of the defocus amount at each position set in the target region is obtained. The image processing device 30 may perform focus lens control of the lens system 11. As the defocus area is adjusted it would be obvious to adjust the size or area of defocus region)
Regarding claim 12 ASUKABE disclose a warning in a case where an area of the object area is smaller than a predetermined value (ASUKABE Fig. 18, paragraph 0231The user can view the size of the circular icon BC displayed as the defocus map image and recognize the defocus amount of each position in the captured image 60 visually and sensually. Fig. 3, 18 and 24 illustrate the digital system of ASUKABE if the defocus area is smaller than certain value and difficult adjust the area it would be obvious to program the system of ASUKABE to provide the defocus is smaller than certain value or too small).
Regarding claim 13 ASUKABE disclose processor outputs the second information so that the second information is displayed in a sub window on the display unit (ASUKABE in Figs. 18 and 24 in block S110 illustrate display defocus map superimpose on the image. In the system of ASUKABE it would be obvious to program the system (Figs. 3 and 5) to display defocus map on the sub-window. The display of the sub-window is conventional).
Regarding claim 14 ASUKABE disclose the processor scales down and outputs the second image so that the second image is displayed in a sub window on the display unit (ASUKABE in Figs. 18 and 24 in block S110 illustrate display defocus map superimpose on the image. In the system of ASUKABE it would be obvious to program the system (Figs. 3 and 5) to scale the size of defocus map using conventional techniques and display the defocus map as sub-window)
Regarding claim 15 ASUKABE disclose processor sets a sub window for display at a position that does not overlap the second information that is not displayed in the sub window (ASUKABE in Figs. 18 and 24 in block S110 illustrate display defocus map superimpose on the image. In the system of ASUKABE it would be obvious to program the system (Figs. 3 and 5) to scale the size of defocus map using conventional techniques and display the defocus map on the display device on the side of the image).
Regarding claim 16 ASUKABA disclose the information processing apparatus and the image sensor (ASUKABE Figs. 1-3, 5-6, 18-24 and note: Figs 1, 23-24 illustrates image sensor and paragraphs 0096-0099, 0103-0106 and 0152-0154 disclose information processing apparatus and image sensor ).
Regarding claim 17 ASUKABE disclose the processor uses autofocus to determine the object area (ASUKABE paragraph 0002 disclose in which a focus lens is automatically focused on any certain point position in a captured image and technologies for F value control in which an amount of light of an imaging surface is automatically controlled to an optimum value. This obviously corresponds processor uses autofocus to determine the object area).
Regarding claim 18 ASUKABE disclose the image sensor captures the object according to a determination that the image sensor is parallel to the object based on the second information (ASUKABE camera control unit 18 controls the whole imaging device 1 by executing a program stored in the ROM, the flash memory, or the like of the memory unit 19. For example, the camera control unit 18 controls an operation of each necessary unit with regard to control of a shutter speed of the imaging element unit 12, instructions of various kinds of signal processing in the camera signal processing unit 13, acquisition of lens information, an imaging operation or a recording operation in response to an operation by the user, starting/ending control of the moving image recording. It would be obvious to program the control unit of the camera such that to control the camera operation such that captures the object according to a determination that the image sensor is parallel to the object based on the second information i.e. defocus map as discussed above)
Regarding claim 20 ASUKABE disclose non-transitory computer-readable storage medium storing a computer program configured to cause a computer to execute the information processing method (ASUKABE Figs 3-5, paragraphs 0152-0153 and 0418-0421).
Communication Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHRAT I SHERALI whose telephone number is (571)272-7398. The examiner can normally be reached Monday-Friday 8:00AM -5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached on 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ISHRAT I. SHERALI
Examiner
Art Unit 2667
/ISHRAT I SHERALI/Primary Examiner, Art Unit 2667
Prosecution Insights