For Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: ‘imaging unit’ in claim 10, ‘first processing apparatus’ and ‘second processing apparatus’ in claim 11, and ‘third apparatus’ in claim 12. Note that ‘image processing apparatus’ in claim 10 does not invoke an interpretation under 112(f) as the following recited ‘memory storing program’ and ‘processor’ also from claim 10 evaluated at prong C are considered to modify the generic placeholder of ‘image processing apparatus’ by providing sufficient structure for performing the claimed function.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function (e.g. [0030] camera as an example of an imaging unit and edge device as an example of a processing apparatus, where edge device as per [0028] must contain a processor, memory, and interface capabilities), and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-10, 13, and 14 are rejected under 35 U.S.C 103 as being unpatentable over Yan (US20200351439) in view of Wang (US20220207680A1).
Regarding claim 1, Yan teaches an image processing apparatus comprising a memory storing program (“one hardware module performs an operation and stores the output of that operation in a memory device to which it is communicatively coupled” [Yan, 0095]) and a processor (“the software is implemented by hardware such as machine of FIG. 8 that includes processors” [Yan, 0086], Yan FIG. 8) that when executing the program, executes processing to adjust (“a portable electronic device with image capturing capabilities is provided with functionalities for automated and/or semi-automated adjustment” [Yan, 0015]), based on a result of analyzing (“the device is configured for processing image data captured by an on-board camera to determine the image metric, and to perform an automated adjustment action based on the determined value of the image metric” [Yan, 0017], where processing image data and performing action based on the resultant determination fulfills the broadest reasonable interpretation of the term ‘analysis’) the captured image ("of one or more image capturing parameters based on an image metric calculated from image data captured by the device" [Yan, 0015], “Turning now to FIG. 4, therein is shown a flow-chart of one example embodiment of performing operation 321 (FIG. 3) for automated adjustment of an image capture parameter, in this example to adjust the ISO settings of the camera” [Yan, 0047], Yan FIGS. 3 & 4, “Other example image capture parameters which can be auto-adjusted based on the image data include, but are not limited to… camera shutter speed based on one or more of an image brightness metric and an image sharpness/blurriness metric… camera exposure settings (e.g., f-stop values) based on one or more of an image brightness metric and an image sharpness/blurriness metric… camera focus settings based, e.g., on an image sharpness/blurriness metric… camera white balance settings based for example on an image colorization metric… camera flash settings based for example on an image brightness metric; and image stabilization settings based on one or more of an image brightness metric and an image sharpness/blurriness metric [Yan, 0051]). Yan does not teach a processor, that when executing the program, executes image quality enhancement processing to reduce an image quality defect. (Note that Yan does not teach ‘parameters related to the image quality enhancement processing’ but because the statement is a disjunction using ‘or’, it holds true since Yan fulfills the first of the two options. Additionally, note that Wang fulfills ‘parameters related to the image quality enhancement processing’ in the event that there is necessity for its fulfillment).
However, Wang teaches a processor, that when executing the program, executes image quality enhancement processing to reduce an image quality defect ("processing the first intermediate image based on a first deep learning network to obtain a first target image, where functions of the first deep learning network include demosaicing (DM) and noise reduction; and performing at least one of brightness enhancement or color enhancement on the first target image to obtain a second target image" [Wang, 0006]), where the listed functions are examples of image quality defect reductions.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included processing one or more captured images with an image quality enhancement algorithm as suggested by Wang, such that the captured image of Yan further subjected to the analyzing of Yan as disclosed is one subjected to enhancement processing, the suggested motivation that such implementation would more comprehensively improve image quality, as the parameter adjustment occurring before image capture and the processing steps post-image capture would further enhance produced images.
Regarding claim 2, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Yan teaches the processing apparatus further executed to calculate correction values of the parameters based on the result of analyzing the captured image subjected to the image quality enhancement processing ("The device can be configured for processing image data captured by an on-board camera to determine the image metric” [Yan, Abstract] where "the image metric comprises a count of successive video frames having a brightness value that transgresses a predefined threshold brightness" [Yan, 0017]) and update the parameters based on the calculated correction values (“the camera controller is configured to…dynamically adjust one or more image-capturing parameters based on processing of the input signals" [Yan, 0034]). Note that in Applicant’s disclosure, the correction value directly corresponds to an amount of necessary camera parameter adjustment. Similarly, Yan discloses “perform[ing] an automated adjustment action based on the determined value of the image metric” [Yan, [0017]).
Regarding claim 3, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Yan as modified by Wang teaches analyzing (see claim 1 for ‘analysis’ as taught by Yan) the captured image subjected to the image quality enhancement processing (see claim 1 for ‘image quality enhancement processing’ as taught by Wang) depends on at least one of a type of an image sensor of the imaging unit (“The adjustable image capture parameters in some embodiments comprise parameters for on-board processing of raw image data captured by a sensor of the camera, e.g., being directed to processing operations performed by the device between (a) the capturing of image data by the camera sensor (e.g., a charge-coupled device)” [Yan, 0016] where ‘type of image sensor’ includes, but is not limited to, a ‘charge-coupled device’), a type of the image quality enhancement processing, a use of deep learning in the image quality enhancement processing, or a scene of the captured image (“FIG. 5 illustrates an example user interface…includes a display of a current scene captured by a camera” [Yan, 0069], Yan FIG. 5 where "In some embodiments that provide for autotuning of parameters based not only on one or more image metrics, but based additionally and/or in combination on a user input attribute" [Yan, 0019] wherein the ‘user interface’ allows for the analysis to depend on the scene provided by the user) (Note that the statement requires ‘at least one’ of the above options and does not necessitate the fulfillment of all of them).
Regarding claim 4, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Yan teaches analyzing the captured image subject to the image quality enhancement processing includes determining whether an image quality defect has occurred in the captured image subjected to the image quality enhancement processing ("the device is configured for processing image data captured by an on-board camera to determine the image metric, and to perform an automated adjustment action based on the determined value of the image metric. In some embodiments, the image metric is an image brightness metric which comprises a brightness value " [Yan, 0017] where ‘image metric’ is calculated from image data of photos taken before the occurrence of editing or parameter adjustment. Furthermore, “Different measures for determining image brightness from a captured image may, instead or in addition, be employed in other embodiments. Some alternative examples for calculating image brightness include, but are not limited to: the number of blacked out pixels in a frame” [Yan, 0044], which is an example of a detected image quality defect), and wherein, if an image quality defect has occurred in the captured image subjected to the image quality enhancement processing, the parameters are adjusted ("to perform an automated adjustment action based on the determined value of the image metric” [Yan, 0017] and “In such cases, automated adjustment of a camera sensitivity parameter (e.g., camera sensor ISO settings) may be automatically or semi-automatically adjusted if the count of threshold-transgressing frames exceed a predetermined frame count threshold" [Yan, 0018]).
Regarding claim 5, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Wang teaches the processing is further executed to adjust the parameters based on a result of analyzing a specific region set as an analysis target of the captured image subjected to the image quality enhancement processing ("In a possible implementation, the preprocessing further includes estimating at least one of a noise intensity distribution map or a sharpening intensity map of the image. The first deep learning network is specifically configured to implement at least one of the following: controlling noise reduction degrees of different regions of the first intermediate image based on the noise intensity distribution map; and controlling sharpening intensity of different regions of the first intermediate image based on the sharpening intensity map" [Wang, 0035]) where the listed implementations are examples of image quality enhancement processing.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the image-based adjustment device taught by Yan to include the region-specific processing implementation disclosed by Wang, the suggested motivation that such implementation would increase user functionality and improve image quality based on increased precision in editing and processing capabilities.
Regarding claim 6, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Yan teaches at least one of a method for calculating the parameters for adjusting the parameters ("In some embodiments that provide for autotuning of parameters based not only on one or more image metrics, but based additionally and/or in combination on a user input attribute" [Yan, 0019]). or an intensity of the adjustment is settable by a user (“It will be appreciated that adjustment of the sensitivity parameter is thus in this example semi-automated, in that the automated adjustment action performed by the smartphone 500 comprises display of the low-light icon 530 for user selection. In some instances, no parameter adjustment is made unless the user actively selects the surfaced icon [Yan, 0071]).
Regarding claim 7, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Wang teaches the image quality enhancement processing includes at least one of noise reduction processing, fog and mist removal processing, or super-resolution processing ("In a possible implementation, the functions of the first deep learning network further include super-resolution (SR) reconstruction, the raw image has a first resolution, the first target image has a second resolution, and the second resolution is greater than the first resolution" [Wang, 0008] and "In this embodiment of this application, a noise reduction degree of each region may be effectively controlled based on a noise characteristic of each region, or sharpening intensity of each region may be adaptively controlled" [Wang, 0036]).
Regarding claim 8, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Yan teaches parameters related to the imaging unit include at least one of an aperture value, shutter speed, sensitivity, focal distance, or white balance calculation mode ("Examples of such image capture parameters include camera sensitivity (e.g., ISO levels), shutter speed, aperture size, and flash settings" [Yan, 0016]).
Regarding claim 9, Yan in view of Wang teaches the image processing apparatus according to claim 1. Additionally, Wang teaches parameters related to the image quality enhancement processing include at least one of correction parameters related to a brightness, color ("the apparatus further includes a color enhancement module and a brightness enhancement module” [Wang, 0243]), contrast (“In a possible implementation, the enhancement module is specifically configured to implement at least one of the following… contrast increase” [Wang, 0061]), dynamic range, or blur of the captured image (“In a possible implementation, the method is applied to the following scenarios: a dark light scenario, a zoom mode, a high dynamic range (HDR) scenario, and a night mode” [Wang, 0042]). Note that the above is in further view of Wang as presented for the case of claim 1.
Regarding claim 10, this claim is the system claim corresponding to the apparatus of claim 1 and is rejected accordingly. Note that a comparable system is disclosed in Yan in FIG. 2. See above for relevant references.
Regarding claim 13, this claim is the method claim corresponding to the apparatus of claim 1 and is rejected accordingly. Note that a comparable method is disclosed in Yan at [0072] and in Fig. 6. See above for relevant references.
Regarding claim 14, this claim is the non-transitory computer readable storage medium claim corresponding to the apparatus of claim 1 and is rejected accordingly. Note that a comparable medium is disclosed in Yan at [0109]. See above for relevant references.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yan in view of Wang, and further in view of Salmasi (US20220256647A1). Yan teaches a second processing ("The various operations of example methods described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented module” refers to a hardware module implemented using one or more processors" [Yan, 0096]. Note that Yan applies specifically to separate apparatuses in communication as “embodiments where the automated image capture control techniques described herein are provided in cooperation with a smartphone or tablet device” [Yan, 0038], “The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across a number of machines” [Yan, 0098])
configured to, based on a result of analyzing the captured image (“According to one aspect of this disclosure, a portable electronic device with image capturing capabilities is provided with functionalities for automated and/or semi-automated adjustment of one or more image capturing parameters based on an image metric calculated from image data captured by the device" [Yan, 0015]) or parameters related to the image quality enhancement processing (Note that Yan does not teach ‘parameters related to the image quality enhancement processing’ but because the statement is a disjunction using ‘or’, it holds true since Yan fulfills the first of the two options. Additionally, note that Wang fulfills ‘parameters related to the image quality enhancement processing’ in the event that there is necessity for its fulfillment). Yan does not teach a first processing apparatus configured to subject a captured image, acquired by an imaging unit, to image quality enhancement processing to reduce an image quality defect.
However, Wang teaches a first processing apparatus configured to subject a captured image, acquired by an imaging unit ("processing the first intermediate image based on a first deep learning network to obtain a first target image” [Wang, 0006]), to image quality enhancement processing for reducing an image quality defect; “where functions of the first deep learning network include demosaicing (DM) and noise reduction and performing at least one of brightness enhancement or color enhancement on the first target image to obtain a second target image" [Wang, 0006]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the image-based adjustment processors in communication taught by Yan to include a device supporting the image processing method disclosed by Wang, the suggested motivation that such implementation would increase the functionality of the apparatus at taught by Yan and improve the quality of the resultant images as compared to those currently produced by either Yan’s disclosure or Wang’s disclosure independently.
Yan as modified by Wang suggests remote processing embodiments but fails to explicitly disclose a first processing apparatus and a second processing apparatus wherein specific processing steps occur remotely, or that each apparatus houses distinct memory storage units. Salmasi, however, discloses remote processing steps between the two distinct apparatuses (“head-mounted device (HMD)… may include various hardware elements, such as a processor, a memory, a display, one or more cameras (e.g., world-view camera, gaze-view camera, etc.), and a wireless interface for connecting with the Internet, a network, or another computing device” [Salmasi, 0042] and ‘The term “edge device” may be used herein to refer to a computing device that includes a programmable processor and communications circuitry for establishing communication links to consumer devices (e.g., smartphones, UEs, IoT devices, etc.) and/or to network components in a service provider, core, cloud, or enterprise network” [Salmasi, 0042]). Note that the communication between these devices is demonstrated in at least FIG. 1 (provided below), specifically referring to the provided ‘edge device’ example of a smartphone.
PNG
media_image1.png
756
616
media_image1.png
Greyscale
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the image-based adjustment processors in communication taught by Yan in view of Wang by allowing for the distinction of multiple independent processing apparatuses in communication as taught by Salmasi, the suggested motivation that such implementation would increase the processing speed and allow for user multitasking by performing different processing steps on independent apparatuses.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yan in view of Wang and Salmasi as applied to claim 11 above, and further in view of Takashi (JP2014146979A). Yan and Wang teach the imaging system of claim 11, but neither disclose a third apparatus configured to, based on positional information of a detected object, control at least one of an orientation or a viewing angle of the imaging unit.
Takashi teaches a third apparatus (note that the specific number of processors used to complete this operation includes, but is not limited to, three, as taught by Yan and disclosed above (see claim 11 rejection)) configured to, based on positional information of a detected object, control at least one of an orientation or a viewing angle of the imaging unit (“The image processing device includes: position detection means for detecting the position of a desired object from an image signal transmitted from the monitor camera (page 21 paragraph 1) and “image processing unit is positioned at the center of the image captured by the imaging unit based on position information of the object detected in the position detection step. An imaging method of an imaging apparatus including a camera control step of sending control signals for controlling each mechanism of pan, tilt or zoom of the imaging unit to the imaging unit” (Takashi, Claim 6 - page 29 paragraph 6)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the previously proposed combination of Yan as modified in view of Wang and Salmasi to comprise a third apparatus to control the orientation or viewing angle of the imaging unit as taught by Takashi. The suggested motivation to create the resultant apparatus is that such implementation might increase the functionality of the camera system to allow for adjustment of camera orientation and field of view parameters in addition to parameters that relate to image quality.
Additional References
Prior art made of record and not relied upon that is considered pertinent to applicant’s disclosure:
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art. Examiner notes that P. Shanthi (“Deep Learning-Based Camera Settings Optimization”), while not prior art due to publishing date, suggests at least all independent claims, as it discloses a method to optimize camera parameters automatically using deep learning, and is included for Applicant’s reference. Examiner also notes that Nourrit (Head-Mounted Miniature Motorized Camera and Laser Pointer Driven by Eye Movements) also suggests at least claim 12, as it discloses a head mounted display that can be controlled remotely to move along the pan/tilt and roll directions (see Introduction and Fig. 2). All other cited references may similarly/alternatively serve to anticipate at least the independent claims as recited, and/or provide examples of current art to Applicant as discovered and noted by Examiner during search.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA T RAMETTA whose telephone number is (571)272-0451. The examiner can normally be reached Monday- Friday, 8 a.m. 5 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JULIA T RAMETTA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights