Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification, as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
In claim 8, “an acquiring unit,” “an image segmenting unit,” “an effect processing unit,” and “an insert object selecting sub-unit.”
In dependent claims 10 and 12-14, each of the claimed units and subunits.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
A review of the specification shows the following structure corresponding to the claim limitations: Specification, fig 10, paragraph 115, lines 1-4, and paragraph 116, describe that the apparatus may be comprised of a processor executing instructions stored in memory, where the instructions implement the claimed functions.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6, 8-11, 13, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Hold-Geoffroy et al. (U.S. Patent Application Publication No. 2018/0359416), referred herein as Hold, in view of Sytnik et al. (U.S. Patent Application Publication No. 2021/0150682), referred herein as Sytnik.
Regarding claim 1, Hold teaches an image processing method comprising: acquiring a first image, and acquiring attribute information of sky in the first image, the attribute information comprising at least one of weather, time, and type of region (fig 1; paragraph 32; paragraph 34, the last 8 lines; paragraph 35, lines 1-2; paragraph 36, lines 7-12; paragraphs 48 and 56; attribute information of the sky is acquired, including weather, time, and/or type of region);
performing sky segmentation processing on the first image to obtain a sky image in the first image, wherein performing sky segmentation processing on the first image to obtain the sky image in the first image comprises identifying a ground boundary in the first image so as to use sky pixel lighting information above the ground boundary in the sky image (fig 3; paragraph 32, the last 6 lines; paragraph 42, lines 1-3; paragraph 43; paragraph 45, lines 1-8; paragraph 46, lines 1-2; paragraph 52, lines 1-7; paragraph 64, lines 1-11; an environment map that comprises an identified horizon [a ground boundary] and sky information above the horizon, is used when processing the sky image); and
performing corresponding effect processing on the sky image in the first image based on the attribute information, wherein the performing corresponding effect processing on the sky image in the first image based on the attribute information comprises: obtaining an insert object matched with the attribute information, and inserting the insert object to the sky image (fig 1; object 112; paragraph 38, lines 1-13; paragraph 64, lines 1-11; paragraph 105; effect processing on the sky image is performed based on the attribute information, comprising obtaining an insert object matched with the attribute information, and inserting the object into the sky image).
Hold teaches identifying a ground boundary (e.g. a horizon) and using the sky information above the ground boundary to generate the sky image (please see the citations above). However, Hold does not explicitly teach using a pixel image above the ground boundary as the sky image.
However, in a similar field of endeavor, Sytnik teaches an image processing method comprising acquiring a first image and attribute information of sky in the first image, and performing sky segmentation processing on the first image to obtain a sky image in the first image (figs 17 and 20; paragraph 73, lines 16-19; paragraph 74; paragraph 77, lines 1-11 and the last 4 lines; paragraph 88, lines 1-4 and 14-25), wherein the sky segmentation comprises identifying a ground boundary so as to use a pixel image above the ground boundary as the sky image (figs 18 and 21-24; paragraph 70, lines 1-7 and the last 8 lines; paragraph 73, the last 2 lines; paragraph 74, lines 1-8; paragraph 78, lines 1-11; paragraph 88, lines 1-4 and 14-25).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the sky pixel image processing of Sytnik with the sky image processing of Hold because this helps to more efficiently perform the replacement and/or modification of the sky with improved accuracy, regardless of image complexity, which is especially important in sky image processing such as that disclosed by Hold (see, for example, Sytnik, paragraphs 3 and 4).
Regarding claim 2, Hold in view of Sytnik teaches the method according to claim 1, wherein the acquiring the attribute information of the sky in the first image comprises: processing the first image based on a preset identification model to obtain the attribute information of the sky in the first image (Hold, paragraph 32, lines 1-7; paragraph 36, lines 7-12; paragraph 48; paragraph 56; Sytnik, paragraph 73, lines 1-7; paragraph 107; the motivation to combine is the same as that discussed in the rejection of claim 1).
Regarding claim 3, Hold in view of Sytnik teaches the method according to claim 1, wherein the acquiring the attribute information of the sky in the first image comprises: acquiring at least one of time when the first image is shot by a shooting device, type of region, orientation, and weather at a located position; and using at least one of the time, the type of region, the orientation, and the weather as the attribute information of the sky (Hold, fig 3; paragraph 32, lines 1-7; paragraph 43; paragraph 48; paragraph 105, lines 1-6; Sytnik, paragraph 92, the last 9 lines; paragraph 107; the motivation to combine is the same as that discussed in the rejection of claim 1).
Regarding claim 4, Hold in view of Sytnik teaches the method according to claim 1, wherein the insert object comprises at least one of an image, animation or AR object (Hold, fig 1; paragraph 38, lines 1-13; paragraph 105; Sytnik, paragraphs 70, 89, 96; the motivation to combine is the same as that discussed in the rejection of claim 1).
Regarding claim 6, Hold in view of Sytnik teaches the method according to claim 4, wherein after inserting the insert object to the sky image, the method further comprises: regulating a brightness of the insert object according to a brightness of an insert position, so that the brightness of the insert object is matched with the brightness of the insert position (Hold, fig 1; paragraph 32; paragraph 34, the last 8 lines; paragraph 35, lines 1-2; paragraph 38, lines 1-13; Sytnik, paragraphs 96-98; the motivation to combine is the same as that discussed in the rejection of claim 1).
Regarding claim 8, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the structural components, which are taught by Hold, fig 12, processor 1202, memory 1204; paragraph 123); thus they are rejected on similar grounds.
Regarding claims 9-11 and 13, the limitations of these claims substantially correspond to the limitations of claims 2-4 and 6, respectively (structural components are taught by Hold, as discussed above in the rejection of claim 8); thus they are rejected on similar grounds as their corresponding claims.
Regarding claim 15, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the memory, processor, and program, which are taught by Hold, fig 12, memory 1204, processor 1202; paragraph 123); thus they are rejected on similar grounds.
Regarding claim 16, Hold teaches a non-transitory computer readable storage medium, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor (fig 12, medium 1204, processor 1202; paragraph 123), the method according to claim 1 is implemented (please see the rejection of claim 1, above).
Claims 5, 7, 12, 14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Hold, in view of Sytnik, and further in view of Izawa et al. (U.S. Patent Application Publication No. 2009/0087123), referred herein as Izawa.
Regarding claim 5, Hold in view of Sytnik teaches the method according to claim 4, wherein the determining the insert object matched with the attribute information based on the attribute information comprises: showing a candidate object matched with the attribute information based on the attribute information; and selecting the candidate object as the insert object, in response to an insertion operation on the candidate object (Hold, fig 1; paragraph 38, lines 1-13; paragraph 105; Sytnik, paragraphs 89 and 96-98; the motivation to combine is the same as that discussed in the rejection of claim 1).
Hold in view of Sytnik teaches providing a database of attributes (Hold, paragraph 96, lines 1-10; Sytnik, fig 3; the motivation to combine is the same as that discussed in the rejection of claim 1), but does not teach showing a plurality of candidate objects, and selecting a candidate object of the plurality of candidate objects as the insert object in response to a selection operation on the candidate object.
However, in a similar field of endeavor, Izawa teaches an image processing method comprising acquiring a first image, identifying sky regions in the first image, and performing effect processing on the sky region (paragraphs 101 and 103; paragraph 104, lines 1-7; paragraphs 105 and 106), wherein the method further comprises showing a plurality of candidate objects, and selecting a candidate object of the plurality of candidate objects as an insert object in response to a selection operation on the candidate object (fig 2, the plurality of candidate objects in the object image database; paragraphs 113 and 114; a plurality of objects are shown, and one of the objects is selected for insertion into the first image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the object selection of Izawa with the object insertion of Hold in view of Sytnik because this makes it possible for custom selection of an object for insertion into the image and allows for changes or corrections to be made as desired, thereby improving both the image processing process and the output image result (see, for example, Izawa, paragraph 115, the last 4 lines).
Regarding claim 7, Hold in view of Sytnik teaches the method according to claim 4, wherein the inserting the insert object to the sky image comprises: acquiring the insert position on the sky image, and inserting the insert object to the insert position on the sky image (Hold, fig 1; paragraph 38, lines 1-13; paragraph 105; Sytnik, paragraphs 88, 89, and 91; the motivation to combine is the same as that discussed in the rejection of claim 1).
Hold in view of Sytnik teaches facilitating user interaction with the image processing system through a user interface (Hold, paragraphs 96 and 126; Sytnik, paragraphs 82 and 89; paragraphs 95 and 97; the motivation to combine is the same as that discussed in the rejection of claim 1), but does not explicitly teach that the position is specified by a user.
However, in a similar field of endeavor, Izawa teaches an image processing method comprising acquiring a first image, identifying sky regions in the first image, and performing effect processing on the sky region (paragraphs 101 and 103; paragraph 104, lines 1-7; paragraphs 105 and 106), wherein the effect processing comprises inserting an object in the first image by acquiring an insert position specified by a user, and inserting the insert object to the insert position specified by the user (paragraphs 105 and 106; paragraph 114; paragraph 115, lines 1-5).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the user position selection of Izawa with the image insertion of Hold in view of Sytnik because this makes it possible for custom selection of an object for insertion into the image and allows for changes or corrections to be made as desired, thereby improving both the image processing process and the output image result (see, for example, Izawa, paragraph 115, the last 4 lines).
Regarding claims 12 and 14, the limitations of these claims substantially correspond to the limitations of claims 5 and 7, respectively; thus they are rejected on similar grounds as their corresponding claims.
Regarding claim 17, Hold in view of Sytnik, further in view of Izawa teaches the method according to claim 5, wherein after inserting the insert object to the sky image, the method further comprises: regulating a brightness of the insert object according to a brightness of an insert position, so that the brightness of the insert object is matched with the brightness of the insert position (Hold, fig 1; paragraph 32; paragraph 34, the last 8 lines; paragraph 35, lines 1-2; paragraph 38, lines 1-13; Sytnik, paragraphs 96-98; the motivation to combine is the same as that discussed in the rejection of claim 1; this subject matter substantially corresponds to that of claim 6).
Regarding claim 18, Hold in view of Sytnik, further in view of Izawa teaches the method according to claim 5, wherein the inserting the insert object to the sky image comprises: acquiring the insert position on the sky image specified by a user; and inserting the insert object to the insert position on the sky image specified by the user (Hold, fig 1; paragraph 38, lines 1-13; paragraph 105; Sytnik, paragraphs 88, 89, and 91; paragraphs 82, 89, 95, and 97; the motivation to combine is the same as that discussed in the rejection of claim 1; Izawa, paragraphs 105 and 106; paragraph 114; paragraph 115, lines 1-5; this subject matter substantially corresponds to that of claim 7; the motivation to combine is the same as that discussed in the rejection of claim 7).
Regarding claims 19 and 20, the limitations of these claims substantially correspond to the limitations of claims 17 and 18, respectively; thus they are rejected on similar grounds as their corresponding claims.
Response to Arguments
On page 9 of the Applicant’s Remarks, with respect to the 103 rejection of claim 1, the Applicant argues that Hold does not teach the amended limitations because the disclosed lighting conditions of Hold is not weather, time, or type of region of the sky in the image. The Examiner respectfully disagrees with this argument.
It is first noted that the claim only requires one of weather, time, or type of region; however, as discussed herein, it is respectfully submitted that the Hold clearly discloses examples of all three of these attributes. The Remarks reference portions of cited paragraphs 35 and 38, but do not address the citations provided that are relevant to the claimed weather, time, or type of region. For example, cited paragraph 32 explicitly states that the lighting attributes may represent lighting conditions at the time in which a digital image was captured [reading on the claimed “time” attribute]; cited paragraphs 36 and 48 describe lighting attributes that may comprise sun position and directional light source parameters, exposure, and wavelength parameters [reading on the broadly claimed “type of region” attribute]; and cited paragraphs 36, 48, and 56 describe lighting attributes comprising atmospheric turbidity [reading on the claimed “weather” attribute]. Thus, the Examiner respectfully submits that Hold clearly teaches the broadly claimed attributes comprising one of weather, time, or type of region.
On pages 9 and 10 of the Applicant’s Remarks, with respect to the 103 rejection of claim 1, the Applicant argues that Sytnik does not cure the deficiencies of Hold because the relighting of Sytnik does not include attribute information that is one of weather, time, and type of region of the sky in the image, and Sytnik does not teach inserting the object. The Examiner respectfully disagrees with this argument.
It is first respectfully submitted that Sytnik is not relied upon to teach the amended claim features, as they are disclosed in Hold (as discussed in the response and rejection above). However, it is additionally noted that the Remarks do not address some of the citations to Sytnik that are relevant to these limitations. For example, cited paragraph 107 of Sytnik explicitly discloses an attribute comprising a type of region, such as “a sunset or other type of sky” which helps determine appropriate colors for object insertion and/or modification; and cited paragraph 70 explicitly discloses the object insertion. Thus, the Examiner respectfully submits that, although Sytnik is not needed to teach these features, Sytnik clearly teaches the broadly claimed attributes comprising one of weather, time, or type of region, as well as object insertion.
On pages 10 and 11 of the Applicant's Remarks, the Applicant argues that the remaining independent claims are not taught by the prior art for reasons similar to those discussed in regard to claim 1, and that the dependent claims are not taught by the prior art, insomuch as they depend from claims that are not taught by the prior art. The Examiner respectfully disagrees with these arguments, for the reasons discussed above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights