DETAILED ACTION
Contents
I. Notice of Pre-AIA or AIA Status 5
II. Priority 5
III. Pertinent Prosecution History 5
IV. Reissue Requirements 6
V. Election/Restrictions 7
VI. Claim Status 10
VII. Information Disclosure Statement 11
VIII. Oath/Declaration 11
IX. Claim Interpretation 13
A. Lexicographic Definitions 14
B. 35 U.S.C. § 112 6th Paragraph 14
X. Claim Rejections – 35 U.S.C. § 112 15
A. 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph 15
XI. Claim Rejections - 35 USC § 251 16
A. Oath/Declaration 16
XII. Claim Rejections – 35 USC § 103 16
A. Claims 1, 3, 5, 13 and 14 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”). 17
B. Claims 2, 4, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”). 22
C. Claims 6 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Thumpudi (U.S. Publication No. 2018/0367734)(“Thumpudi”). 24
D. Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”). 28
E. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) and Shikata (U.S. Publication No. 2016/0381302)(“Shikata”) as applied to claims 7 and 8 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”). 30
F. Claims 10 and 12 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Tsutsumi U.S. Patent No. 9,025,038) (“Tsutsumi”). 32
G. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 10 and 12 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”). 38
H. Claims 1, 3, 5, 13 and 14 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”). 41
I. Claims 2, 4, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”). 47
J. Claims 6 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Thumpudi (U.S. Publication No. 2018/0367734)(“Thumpudi”). 49
K. Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”). 53
L. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) and Shikata (U.S. Publication No. 2016/0381302)(“Shikata”) as applied to claims 7 and 8 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”). 55
M. Claims 10 and 12 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”). 57
N. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 10 and 12 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”).. 65
XIII. Conclusion 68
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Applicant filed the instant reissue application 18/300,117 (“‘117 Reissue Application”) on 13 September 2023 for U.S. Application No. 16/527,563 (“‘563 Application”), filed 31 July 2019, now U.S. Patent No. 11,182,887 (“‘887 Patent”), issued 23 November 2021, which claims foreign priority to Korean Patent Application No. 10-2019-0001728, filed 07 January 2019 (“KPA ‘728”).
Thus, the Examiner concludes that for examination purposes the instant ‘117 Reissue Application has an effective filing data of 07 January 2019, which is the filing date of the KPA ‘728 Application.
Pertinent Prosecution History
As set forth supra, Applicant filed the application for the instant ‘117 Reissue Application on 13 September 2023. The Examiner finds that the instant ‘117 Reissue Application included a preliminary amendment (“Sept 2023 Preliminary Amendment”) to the claims (“Sept 2023 Claim Amendment”).
Reissue Requirements
For reissue applications filed before September 16, 2012, all references to 35 U.S.C. 251 and 37 CFR 1.172, 1.175, and 3.73 are to the law and rules in effect on September 15, 2012. Where specifically designated, these are “pre-AIA ” provisions.
For reissue applications filed on or after September 16, 2012, all references to 35 U.S.C. 251 and 37 CFR 1.172, 1.175, and 3.73 are to the current provisions.
Applicant is reminded of the continuing obligation under 37 CFR 1.178(b), to timely apprise the Office of any prior or concurrent proceed-ing in which the ‘887 Patent is or was involved. These proceedings would include interferences, reissues, reexaminations, post-grant proceedings and litigation.
Applicant is further reminded of the continuing obligation under 37 CFR 1.56, to timely apprise the Office of any information which is mate-rial to patentability of the claims under consideration in this reissue appli-cation.
These obligations rest with each individual associated with the filing and prosecution of this application for reissue. See also MPEP §§ 1404, 1442.01 and 1442.04.
The Examiner notes that Amendment practice for Reissue Applications is NOT the same as for non-provisional applications. See MPEP §§ 1413 and 1453. Reissue application amendments must comply with 37 CFR 1.173, while non-provisional application amendments must comply with 37 CFR 1.121. Particularly,
Manner of making amendments under 37 CFR 1.173:
All markings (underlining and bracketing) are made relative to the original patent text, 37 CFR 1.173(g) (and not relative to the prior amendment).
For amendments to the abstract, specification and claims, the deleted matter must be enclosed in brackets, and the added matter must be underlined. See 37 CFR 1.173(d).
For amendments to the drawings, any changes to a patent drawing must be submitted as a replacement sheet of drawings which shall be an attachment to the amendment document. Any replacement sheet of drawings must be in compliance with § 1.84 and shall include all of the figures appearing on the original version of the sheet, even if only one figure is amended. Amended figures must be identified as "Amended," and any added figure must be identified as "New." In the event that a figure is canceled, the figure must be surrounded by brackets and identified as "Canceled." All changes to the drawing(s) shall be explained, in detail, beginning on a separate sheet accompanying the papers including the amendment to the drawings. See 37 CFR 1.173(d)(3).
The Examiner further notes that all amendments to the instant ‘117 Reissue Application must comply with 37 CFR 1.173(b)-(g).
Election/Restrictions
Applicable rules and regulations:
CFR 1.176 states (in pertinent parts):
(b) Restriction between subject matter of the original patent claims and previously unclaimed subject matter may be required (restriction involving only subject matter of the original patent claims will not be required). If restriction is required, the subject matter of the original patent claims will be held to be constructively elected unless a disclaimer of all the patent claims is filed in the reissue application, which disclaimer cannot be withdrawn by applicant.
[42 FR 5595, Jan. 28, 1977; revised, 65 FR 54604, Sept. 8, 2000, effective Nov. 7, 2000].
MPEP § 1450 states (in pertinent parts):
37 CFR 1.176(b) permits the examiner to require restriction in a reissue application between claims newly added in a reissue application and the original patent claims, where the added claims are directed to an invention which is separate and distinct from the invention(s) defined by the original patent claims. The criteria for making a restriction requirement in a reissue application between the newly added claims and the original claims are the same as that applied in a non-reissue application. See MPEP §§ 806 through 806.05(i). The authority to make a "restriction" requirement under 37 CFR 1.176(b) extends to and includes the authority to make an election of species.
Where a restriction requirement is made by the examiner, the original patent claims will be held to be constructively elected (except for the limited situation where a disclaimer is filed as discussed in the next paragraph). Thus, the examiner will issue an Office action in the reissue application (1) providing notification of the restriction requirement, (2) holding the added claims to be constructively non-elected and withdrawn from consideration, (3) treating the original patent claims on the merits, and (4) informing applicant that if the original claims are found allowable, and a divisional application has been filed for the non-elected claims, further action in the application will be suspended, pending resolution of the divisional application.
If a disclaimer of all the original patent claims is filed in the reissue application containing newly added claims that are separate and distinct from the original patent claims, only the newly added claims will be present for examination. In this situation, the examiner's Office action will treat the newly added claims in the reissue application on the merits. The disclaimer of all the original patent claims must be filed in the reissue application before the issuance of the examiner's Office action containing the restriction requirement, in order for the newly added claims to be treated on the merits. Once the examiner has issued the Office action providing notification of the restriction requirement and treating the patent claims on the merits, it is too late to obtain an examination on the added claims in the reissue application by filing a disclaimer of all the original patent claims, If reissue applicant wishes to have the newly added claims be treated on the merits, a divisional reissue application must be filed to obtain examination of the added claims. Reissue applicants should carefully note that once a disclaimer of the patent claims is filed, it cannot be withdrawn. It does not matter whether the reissue application is still pending, or whether the reissue application has been abandoned or issued as a reissue patent. For all these situations 37 CFR 1.176(b) states that the disclaimer cannot be withdrawn; the disclaimer will be given effect. Note that cancellation of all the original patent claims in the reissue application will not be effective as an alternative to disclaiming all the original patent claims, and 37 CFR 1.176(b) will not be waived to permit the same. This is because the patent owner can subsequently file a reissue continuation presenting the original patent claims.
Restriction between multiple inventions recited in the newly added claims will be permitted provided the added claims are drawn to several separate and distinct inventions. In such a situation, the original patent claims would be examined in the first reissue application, and applicant is permitted to file a divisional reissue application for each of the several separate and distinct inventions identified in the examiner's restriction requirement.
A situation will sometimes arise where the examiner makes an election of species requirement between the species claimed in the original patent claims and a species of claims added in the reissue application. (The filing of a reissue application to only add species claims that require all the limitations of an issued generic claim would not meet the requirements of 35 U.S.C. 251 - see MPEP § 1402; however, this situation can occur where there is another change to the patent being made, which does correct a 35 U.S.C. 251 "error." ) In such a situation, if (1) the non-elected claims to the added species depend from (or otherwise include all limitations of) a generic claim which embraces all species claims, and (2) the generic claim is found allowable, then the non- elected claims of the added species must be rejoined with the elected claims of the original patent. See MPEP § 821.04(a).
MPEP 1450.
This application contains claims directed to the following patentably distinct species: Species I – Figures 1-12 (i.e., claims 1-12, 13, 14, 16, 18 and 20); Species II Figures 13-18 (i.e., claims 15, 17, 19 and 21-27); and Species III Figure 19 (i.e., claims 28-35). The species are independent or distinct because the claims to the different species recite the mutually exclusive characteristics of such species. In addition, these species are not obvious variants of each other based on the current record.
Currently, claims are 13, 14, 16, 18 and 20 are generic.
There is a serious search and/or examination burden for the patentably distinct species as set forth above because at least the following reason(s) apply: the species or groupings of patentably indistinct species require a different field of search (e.g., searching different classes/subclasses or electronic resources, or employing different search strategies or search queries).
Restriction for examination purposes as indicated is proper because all the inventions listed in this action are independent or distinct for the reasons given above and there would be a serious search and/or examination burden if restriction were not required because one or more of the following reasons apply: the inventions require a different field of search (e.g., searching different classes/subclasses or electronic resources, or employing different search strategies or search queries).
Should applicant traverse on the ground that the species, or groupings of patentably indistinct species from which election is required, are not patentably distinct, applicant should submit evidence or identify such evidence now of record showing them to be obvious variants or clearly admit on the record that this is the case. In either instance, if the examiner finds one of the species unpatentable over the prior art, the evidence or admission may be used in a rejection under 35 U.S.C. 103 or pre-AIA 35 U.S.C. 103(a) of the other species.
Since no disclaimer of all the original patent claims 1-12 has been filed before the issuance of this Office action in the instant reissue application, the original patent claims 1-12 and, new claims 13, 14, 16, 18 and 20, will be held to be constructively elected by original presentation for prosecution on the merits, and the newly added claims 15, 17, 19 and 21-35. are withdrawn from consideration as being directed to a non-elected inventions.
Claim Status
The Examiner finds that the claim status in the instant ‘117 Reissue Application is as follows:
Claim(s) 1-12 (Original)
Claim(s) 13, 14, 16, 18 and 20 (New)
Claim(s) 15, 17, 19 and 21-35 (New and Withdrawn)
Thus, the Examiner concludes that claims 1-35 are pending; and claims 15, 17, 19 and 21-35 are withdrawn in the instant ‘117 Reissue Application. Thus, claims 1-14, 16, 18 and 20 are examined (“Examined Claims”).
Information Disclosure Statement
The Applicants’ Information Disclosure Statements filed: 13 April 2023 (April 2023 IDS”); 07 June 2023 (June 2023 IDS”); and 06 July 2023 (July 2023 IDS”) have been received and entered into the record. Since the Information Disclosure Statements comply with the provisions of MPEP § 609, the references cited therein have been considered by the Examiner. See attached form PTO-1449.
It is noted that several foreign language documents were included in the April 2023 IDS and July 2023 IDS. In these cases, the Office has annotated the entries in the form 1449 where applicable to indicate the degree to which the document has been considered (limited to English language disclosures).
Oath/Declaration
The Examiner finds that the Declaration filed by Applicant on 13 April 2023 (“April 2023 Oath/Declaration”) is defective because of the following:
The Examiner finds that the April 2023 Oath/Declaration filed by Applicant states,
This reissue application is a broadening reissue application, claims 1-12 include independent claims 1 and 10 directed to a method of image processing and an image processing device, respectively, where, in both claim sets, the second HDR image is based on the second frame image and a third frame image.
With respect to the original claims, one error was the failure to appreciate the benefit of pursuing linking claims that are directed to a method of imaging processing and an image processing device, where, in both claim sets, the second HDR image is based on a reference image and the third frame image, where dependents further define that the reference image may be either the second frame image or the first HDR image New claims 13-27 address this error. Further, another error was the failure to appreciate disclosed but overlooked embodiments such as claims directed to the embodiment of FIG. 19, in which an image processing device includes multiple HDR processors and frame buffers. New claims 28-35 address this error.
(April 2023 Oath/Declaration; emphasis added). Thus, Applicant asserts that new claims 13-27 are to linking claims; and 28-35 are overlooked embodiment claims. While the April 2023 Oath/Declaration states that the instant ‘117 Reissue Application is a broadening application, the April 2023 Oath/Declaration is defective because the statement does not specify a particular error within the ‘887 Patent as a basis for the instant ‘117 Reissue Application.
First, the April 2023 Oath/Declaration filed with this application is defective because the error which is relied upon to support the reissue application is not an error upon which a reissue can be based. See 37 CFR 1.175 and MPEP § 1414.
The Examiner finds that the error upon which the instant reissue is based is addressed partially by the non-elected Species II Figures 13-18 (i.e., claims 13-27); and Species III (i.e., claims 28-35). From this perspective, the claims, as set forth below, that are linking claims with respect to Species I (i.e., claims 1-12), which is constructively elected by original presentation, are included in the instant ‘117 Reissue Application. Subsequently, the claims, as set forth below, that are overlooked aspect claims with respect to Species I (i.e., claims 1-12), which is constructively elected by original presentation, are not included in the instant ‘117 Reissue Application. Thus, the April 2023 Oath/Declaration is incorrect because it does not correctly state an error upon which this instant ‘117 Reissue Application is based.
In addition, a proper error statement must identify a single word, phrase, or expression in the specification or in an original claim in the underlying patent, i.e., the ‘887 Patent, and how it renders the original patent wholly or partly inoperative or invalid. (See MPEP §1414(II)).
Thus, Applicant is required to provide a new declaration with: (1) a statement of error with respect to ‘887 Patent, directed to a correction with respect to the elected claims above; and (2) identifying “a single word, phrase, or expression” from the ‘887 Patent that was not included therein that rendered the ‘887 Patent invalid or inoperative. (See 37 CFR 1.175 and MPEP § 1414).
Claim Interpretation
During examination, claims are given the broadest reasonable interpretation consistent with the specification and limitations in the specification are not read into the claims. See MPEP § 2111, MPEP § 2111.01 and In re Yamamoto et al., 222 USPQ 934 (Fed. Cir. 1984). Under a broadest reasonable interpretation, words of the claim must be given their plain meaning, unless such meaning is inconsistent with the specification. See MPEP § 2111.01(I). It is further noted it is improper to import claim limitations from the specification, i.e., a particular embodiment appearing in the written description may not be read into a claim when the claim language is broader than the embodiment. See MPEP § 2111.01(II). Therefore, unless one of the exceptions applies below, Examiners will interpret the limitations of the pending and examined claims using the broadest reasonable interpretation.
Lexicographic Definitions
A first exception to the prohibition of reading limitations from the specification into the claims is when the Applicant for patent has provided a lexicographic definition for the term. (See MPEP § 2111.01(IV)). After careful review of the original specification, the prosecution history, and unless expressly noted otherwise by the Examiner, the Examiner finds that he is unable to locate any lexicographic definitions (either express or implied) with reasonable clarity, deliberateness, and precision. Because the Examiner is unable to locate any lexicographic definitions with reasonable clarity, deliberateness, and precision, the Examiner concludes that Applicant is not his/her own lexicographer. (Id.)
35 U.S.C. § 112 6th Paragraph
A second exception to giving words in the claims their ordinary and customary meaning is when a claimed phrase is interpreted in accordance with 35 U.S.C. § 112 6th paragraph. See MPEP § 2181 et seq.
The Examiner finds that because the Examined Claims do not recite “step,” “means” or a claim term used as a substitution for “means” (i.e. a generic placeholder for “means”), the Examined Claims fail Prong (A) as set forth in MPEP §2181. Because the twenty-four (24) Examined Claims fail Prong (A) as set forth in MPEP §2181 I., the Examiner concludes that all Examined Claims do not invoke 35 U.S.C. §112, 6th paragraph. See also Ex parte Miyazaki, 89 USPQ2d 1207, 1215-16 (B.P.A.I. 2008)(precedential).
Claim Rejections – 35 U.S.C. § 112
35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 13, 14, 16, 18 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to the limitations of claim 13, the Examiner finds that claim13, in part, recites the limitations,
… combining the shifted first frame image and the second frame image, the second frame image being subsequent in time to the first frame image …; and
… combining the shifted reference frame image and the third frame image, the third frame image being subsequent in time to the second frame image …; and
(Sept 2023 Claim Amendment at claim 13; emphasis added). Applicant has asserted that claim 13 is a linking claim with respect to the original claim 1. (See April 2023 Oath/Declaration). In examination, of original claim 1 of the 887 Patent the Examiner finds that claim13, in part, recites the limitations,
… combining the shifted first frame image and the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images …;
… combining the shifted second frame image and the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images …; and
(‘887 Patent at claim 1; emphasis added). It is unclear and indefinite to exactly which first/second frame image the second/third frame image is subsequent in to. Further clarification is required.
The Examiner recommends that claim 13 recite the claim requirement exactly the same as claim 1 noted above.
Claims 14, 16, 18 and 20 are rejected in light of their dependency from at least independent claim 13.
Claim Rejections - 35 USC § 251
Oath/Declaration
Claims 1-35 are rejected as being based upon a defective reissue declaration under 35 U.S.C. 251 as set forth above. See 37 CFR 1.175.
The nature of the defect(s) in the declaration is set forth in the discussion above in this Office action. (See §§ V; VIII supra).
Claim Rejections – 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5, 13 and 14 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”).
With respect to the limitations of claims 1 and 13, and
[1, 13] [a] method of image processing a video image including a plurality of frame images, the method comprising:
In this regard, the Examiner finds that Yamaguchi discloses an image processing system and method for processing a video image including a plurality of frame images. (Yamaguchi at Figures 1, 2; see mixer 6/frame memory 7 combination).
generating a first high dynamic range (HDR) image based on a first frame image and a second frame image of the plurality of frame images by (i) determining an amount of movement between the first frame image and the second frame image, (ii) shifting the first frame image or an object therein based on the amount of movement to generate a shifted first frame image, and (iii) combining the shifted first frame image and the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images, the first frame image having a first exposure time, the second frame image having a second exposure time different from the first exposure time; and
generating a second HDR image based on the second frame/reference image and a third frame image of the plurality of frame images by (i) determining an amount of movement between the second frame image and the third frame image, (ii) shifting the second frame image or an object therein based on the amount of movement to generate a shifted second frame image, and (iii) combining the shifted second frame image and the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images, the third frame image having the first exposure time
In this regard, the Examiner finds that Yamaguchi discloses the system and method generating: (1) a first high dynamic range (HDR) image based on a first frame image (Yamaguchi at Figure 2; -see the “nth frame”) and a second frame image (id.; see the (n+1)th frame) of the plurality of frame images by (iii) combining the first frame image and second frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]); and (2) a second HDR image based on the second frame image and a third frame image (id. at Figure 2; see the (n+2)th frame) of the plurality of frame image by (iii) combining the second frame image and third frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]). The Examiner finds that: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id.)
Yamaguchi discloses the limitations, as previously set forth, except for specifically calling for generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image.
However, generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image is known in the art.
The Examiner find that Tsutsumi teaches, for example, an image capturing apparatus and method that captures images at various exposure times (Tsutsumi at c.5, ll.24-28; c.8, ll.15-40; c.8, l.56 – c.9, l.11; c.9, ll.49-51; Figures 10, 11) in which, before images of different exposures time are combined to form an image, the generating further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image. (Id. at Abstract; c.8, l.56 – c.9, l.11; c.9, l.13 – c.11, l.67; see Figure 12-14).
The Examiner finds that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image as described in Tsutsumi in the system and method for processing a video image of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image, since it provides a mechanism to compensate for camera/sensor shake. (Id. at Abstract).
With respect to the limitations of claim 3, and
[3] further comprising: generating a third HDR image based on the third frame image and a fourth frame image of the plurality of frame images, the fourth frame image being subsequent in time to the third frame image in the plurality of frame images, the fourth frame image having the second exposure time.
In this regard, the Examiner finds that Yamaguchi discloses the system and method generating a third HDR image based on the third frame image and a fourth frame image (Yamaguchi at Figure 2; see (n+3)th frame is a long-exposure image) of the plurality of frame images, the fourth frame image being subsequent in time to the third frame image in the plurality of frame images, the fourth frame image having the second exposure time (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the n+2th frame and the (n+3)th frame to create a third HDR image). The Examiner finds that: (1) the (n+2)th frame and the (n+3)th frame shown in Figure 2 correspond to “a third frame image” and “fourth frame image,” respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id.)
With respect to the limitations of claim 5, and
[5] wherein the determining the amount of movement includes: extracting a global movement between the first frame image and the second frame image.
In this regard, the Examiner finds that Tsutsumi teaches the amount of movement being a global movement between first and second frame images. (Tsutsumi at Abstract; c.9, ll.28-42; c.10, l.47 -c.11, l.53).
Thus, similarly, as set forth above, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determining the amount of movement to include extracting a global movement between the first frame image and the second frame image as described by Tsutsumi in the system and method for processing a video image of Yamaguchi- and Tsutsumi.
A person of ordinary skill in the art would be motivated to incorporate the determining the amount of movement to include extracting a global movement between the first frame image and the second frame image, since it provides a mechanism to compensate for camera/sensor shake. (Id. at Abstract).
With respect to the limitations of claim 14, and
[14] wherein the reference image is one of the second frame image or the first HDR image.
In this regard, the Examiner finds that Yamaguchi discloses the system and method generating: (1) a first high dynamic range (HDR) image based on a first frame image (Yamaguchi at Figure 2; -see the “nth frame”) and a second frame image (id.; see the (n+1)th frame) of the plurality of frame images. (Yamaguchi at c.9, ll.28-42; see Figure 2).
Claims 2, 4, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”).
With respect to the limitations of claims 2, 4, 16 and 18, and
[2, 16] wherein a number of the plurality of frame images received before the image processing is equal to a number of a plurality of HDR images obtained after the image processing, the first HDR image and the second HDR image being included in the plurality of HDR images (claims 2 and 16); and
[4, 18] further comprising: outputting a start frame image as a start HDR image without performing the image processing on the start frame image, the start frame image being at a head of the plurality of frame images (claims 4 and 18).
In this regard, the Examiner finds that Yamaguchi discloses the first HDR image and the second HDR image being included in the plurality of HDR images. (Yamaguchi at c.9, ll.37-42; see Figure 2). The Examiner finds that the given the fact that two input frames are required to generate a HDR image, the total number of HDR images will be one less than the total number of input frames. While Yamaguchi discloses the limitations, as set forth above, the Examiner finds that Yamaguchi and Tsutsumi is silent on how to treat the first frame.
However, creating an HDR image from either a sequence of images (i.e., bracketing) or from a single image is known in the art. The Examiner finds that Herman, for example, teaches that an HDR image can be created from either a sequence of images (i.e., bracketing) or from a single image. (Herman at c.5, l.67 – c.6, l.6).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate creating an HDR image from a single first frame image as described by Herman in the system and method for processing video images of Yamaguchi and Tsutsumi.
A person of ordinary skill in the art would be motivated to incorporate creating an HDR image from a single first frame image, since it provides a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Herman to the system and method for processing a video image of Yamaguchi and Tsutsumi would have yielded predicable results and resulted in an improved system, namely, providing creating an HDR image from a single first frame image, in Yamaguchi and Tsutsumi to provide a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Furthermore, and from this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of utilizing a system and method for processing video images, as described in Herman, in order to provide a mechanism to easily create HDR images.
(2) A finding that Herman provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images to create an HDR image by either (i.e., 1) bracketing a plurality of images; or 2) utilizing a single image).
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since Herman explicitly teaches the utilization of a system and method for processing video images to create an HDR image, with either using the bracketing a plurality of images or a single image to create an HDR image, Herman teaches that one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397.
Claims 6 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Thumpudi (U.S. Publication No. 2018/0367734)(“Thumpudi”).
With respect to the limitations of claims 6 and 20, and
[6] wherein the determining the amount of movement includes extracting a local movement for the object between the first frame image and the second frame image, the object being included in both the first frame image and the second frame image (claim 6); and
[20] wherein the amount of movement is a local movement for the object, and the shifting includes shifting the object in the reference image based on the local movement (claim 20).
In this regard, the Examiner finds that Tsutsumi teaches the amount of movement being a global movement between first and second frame images. (Tsutsumi at Abstract; c.9, ll.28-42; c.10, l.47 -c.11, l.53).
The Examiner finds that Yamaguchi and Tsutsumi discloses the limitations, as previously set forth, except for specifically calling for the determining the amount of movement to include extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement.
However, a determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement is known in the art. The Examiner finds that Thumpudi, for example, teaches a system and method for detecting an amount of movement between a series of frames by utilizing global movement detection and/or local movement detection of an object. (Thumpudi at Abstract; ¶¶ 0005-0007; 0014-0015; 0019; 0023-0027; 0042-0043; 0047-0048; claim 1).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate the determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement as described by Thumpudi in the system and method for processing video images of Yamaguchi and Tsutsumi.
A person of ordinary skill in the art would be motivated to incorporate the determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement, since it provides a mechanism to process less information to detect motion. (Id. at ¶¶ 0027; 0043).
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Thumpudi to the system and method for processing a video image of Yamaguchi and Tsutsumi would have yielded predicable results and resulted in an improved system, namely, providing the determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement, in Yamaguchi and Tsutsumi to provide a mechanism to process less information to detect motion.
Furthermore, and from this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of utilizing a system and method for processing video images, as described in Thumpudi, in order to provide a mechanism to process less information to detect motion.
(2) A finding that Thumpudi provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images to detect motion in an image by either ((i.e., 1) extracting global movement; or 2) extracting local movement).
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since Thumpudi explicitly teaches the utilization of a system and method for processing video images to create an HDR image, with either using the extracting global movement or extracting local movement to detect motion, Thumpudi teaches that one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing extracting local movement to detect motion) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing extracting local movement to detect motion) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397.
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”).
With respect to the limitations of claims 7 and 8 and
[7] wherein the generating the first HDR image generates the first HDR image based on the first frame image, the second frame image and a fourth frame image of the plurality of frame images between the second frame image and the third frame image, the fourth frame image having a third exposure time different from the first exposure time and the second exposure time, and the generating the second HDR image generates the second HDR image based on the second frame image, the third frame image, and the fourth frame image (claim 7); and
[8] further comprising: generating a third HDR image based on the third frame image, the fourth frame image and a fifth frame image of the plurality of frame images, the fifth frame image being subsequent in time to the third frame image in the plurality of frame images, the fifth frame image having the second exposure time (claim 8); and
The Examiner finds that Yamaguchi and Tsutsumi discloses the limitations, as previously set forth, except for specifically calling for the frame number/exposure time combinations as set forth above to generate respective HDR images that correlate to the combinations.
However, generating HDR images based upon frame number/exposure time combinations as set forth in the claims above is known in the art. The Examiner finds that Yamaguchi, for example, teaches mixing three, instead of two, image frames while shifting one frame at a time with the method being applicable to image signals recorded with three or more different exposure times. (Yamaguchi at c.7, ll.16-19). Similarly, in the same field of endeavor, the Examiner finds that Shikata further teaches combining three or more successive frames of different exposure times to create respective HDR images. (Shikata at ¶ 0115).
Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above from the teaching of Yamaguchi and Shikata.
A person of ordinary skill in the art would be motivated to incorporate c the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, since it provides a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Yamaguchi and Shikata to the system and method for processing a video image of Yamaguchi and Tsutsumi would have yielded predicable results and resulted in an improved system, namely, providing the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, in Yamaguchi and Tsutsumi to provide a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) and Shikata (U.S. Publication No. 2016/0381302)(“Shikata”) as applied to claims 7 and 8 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”).
With respect to the limitations of claim 9, and
[9] further comprising: outputting a start frame image as a start HDR image without performing the image processing, the start frame image being at a head of the plurality of frame images.
In this regard, the Examiner finds that Yamaguchi discloses the first HDR image and the second HDR image being included in the plurality of HDR images. (Yamaguchi at c.9, ll.37-42; see Figure 2). The Examiner finds that the given the fact that two input frames are required to generate a HDR image, the total number of HDR images will be one less than the total number of input frames. While Yamaguchi discloses the limitations, as set forth above, the Examiner finds that Yamaguchi, Tsutsumi and Shikata is silent on how to treat the first frame.
However, creating an HDR image from either a sequence of images (i.e., bracketing) or from a single image is known in the art. The Examiner finds that Herman, for example, teaches that an HDR image can be created from either a sequence of images (i.e., bracketing) or from a single image. (Herman at c.5, l.67 – c.6, l.6).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate creating an HDR image from a single first frame image as described by Herman in the system and method for processing video images of Yamaguchi, Tsutsumi and Shikata.
A person of ordinary skill in the art would be motivated to incorporate creating an HDR image from a single first frame image, since it provides a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Herman to the system and method for processing a video image of Yamaguchi, Tsutsumi and Shikata would have yielded predicable results and resulted in an improved system, namely, providing creating an HDR image from a single first frame image, in Yamaguchi, Tsutsumi and Shikata to provide a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Furthermore, and from this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of utilizing a system and method for processing video images, as described in Herman, in order to provide a mechanism to easily create HDR images.
(2) A finding that Herman provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images to create an HDR image by either ((i.e., 1) bracketing a plurality of images; or 2) utilizing a single image).
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since Herman explicitly teaches the utilization of a system and method for processing video images to create an HDR image, with either using the bracketing a plurality of images or a single image to create an HDR image, Herman teaches that one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397.
Claims 10 and 12 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Tsutsumi U.S. Patent No. 9,025,038) (“Tsutsumi”).
With respect to the limitations of claim 10, and
[10] [a]n image processing device configured to perform image processing on a video image including a plurality of frame images, the plurality of frame images including a first frame image, a second frame image and a third frame image, the image processing device comprising:
In this regard, the Examiner finds that Yamaguchi discloses an image processing system and method for processing a video image including a plurality of frame images. (Yamaguchi at Figures 1, 2; see mixer 6/frame memory 7 combination). The Examiner finds that Yamaguchi discloses: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively. (Id. at c.9, ll.28-42; and see Figure 2).
a first frame buffer configured to:
receive the first frame image, the first frame image having a first exposure time, and
receive the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images, the third frame image having the first exposure time
a second frame buffer configured to:
receive the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images, the second frame image having a second exposure time different from the first exposure time; and
In this regard, the Examiner finds that Yamaguchi discloses the image processing system comprising a memory unit 802 that is configured to receive the first frame image (Yamaguchi at Figure 2; -see the “nth frame”), the first frame image having a first exposure time, and receive the third frame image (id. at the (n+2)th frame), the third frame image being subsequent in time to the second frame image (id. at the (n+1)th frame) in the plurality of frame images, the third frame image having the first exposure time. (Id. at c.9, ll.28-42; and see Figure 2). In addition, the Examiner finds that the memory unit 802 being further configured to receive the second frame image (id. at the (n+1)th frame), the second frame image being subsequent in time to the first frame image (id. at the “nth frame” in the plurality of frame images. (Id.) The Examiner finds that: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id.)
Yamaguchi discloses the limitations, as previously set forth, except for specifically calling for the memory to comprise a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image.
However, a memory comprising a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image is known in the art. The Examiner finds that Tsuzuki, for example, teaches an image processing apparatus and method comprising a frame memory 20/20A comprising a short term exposure image buffer and a long exposure image buffer with the short term exposure image buffer configured to receive the first and third frame images and a long exposure image buffer configured to receive the second frame image. (Tsuzuki at c.5, l.47 – c.6, l.13; see Figures 1, 2).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate a memory comprising a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image as described by Tsuzuki in the system and method for processing video images of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate a memory comprising a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image, since it provides a mechanism to continuously capture and write images of different exposures of the same corresponding regions. (Id. at c.6, ll.5-10).
a high dynamic range (HDR) processor configured to:
generating a first HDR image based on the first frame image and the second frame image of the plurality of frame images by (i) determining an amount of movement between the first frame image and the second frame image, (ii) shifting the first frame image or an object therein based on the amount of movement to generate a shifted first frame image, and (iii) combining the shifted first frame image and the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images, the first frame image having a first exposure time, the second frame image having a second exposure time different from the first exposure time; and
generating a second HDR image based on the second frame image and the third frame image of the plurality of frame images by (i) determining an amount of movement between the second frame image and the third frame image, (ii) shifting the second frame image or an object therein based on the amount of movement to generate a shifted second frame image, and (iii) combining the shifted second frame image and the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images, the third frame image having the first exposure time
In this regard, the Examiner finds that Yamaguchi discloses the system and method comprising a CPU having memory and programs thereon (Yamaguchi at c.21, ll.45-58) to perform the operations of generating: (1) a first high dynamic range (HDR) image based on a first frame image (Yamaguchi at Figure 2; -see the “nth frame”) and a second frame image (id.; see the (n+1)th frame) of the plurality of frame images by (iii) combining the first frame image and second frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]); and (2) a second HDR image based on the second frame image and a third frame image (id. at Figure 2; see the (n+2)th frame) of the plurality of frame image by (iii) combining the second frame image and third frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]). The Examiner finds that: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id).
Yamaguchi and Tsuzuki discloses the limitations, as previously set forth, except for specifically calling for generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image.
However, generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image is known in the art. The Examiner find that Tsutsumi teaches, for example, an image capturing apparatus and method that captures images at various exposure times (Tsutsumi at c.5, ll.24-28; c.8, ll.15-40; c.8, l.56 – c.9, l.11; c.9, ll.49-51; Figures 10, 11) in which, before images of different exposures time are combined to form an image, the generating further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image. (Id. at Abstract; c.8, l.56 – c.9, l.11; c.9, l.13 – c.11, l.67; see Figure 12-14).
The Examiner finds that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image as described in Tsutsumi in the system and method for processing a video image of Yamaguchi and Tsuzuki.
A person of ordinary skill in the art would be motivated to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image, since it provides a mechanism to compensate for camera/sensor shake. (Id. at Abstract).
With respect to the limitations of claim 12, and
[12] wherein a number of frame buffers that are included in the image processing device is equal to a number of exposure times that are included in the plurality of frame images.
In this regard, the Examiner finds that Yamaguchi discloses he image processing system comprising a memory unit 802 that is configured to receive both short and long exposure times. (Yamaguchi. at c.9, ll.28-42; and see Figure 2).
In addition, the Examiner finds that Tsuzuki, for example, teaches an image processing apparatus and method comprising a number of frame buffers that are included in the image processing device being equal to a number of exposure times that are included in the plurality of frame images. (Tsuzuki at c.5, l.47 – c.6, l.13; see Figures 1, 2).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate a number of frame buffers that are included in the image processing device is equal to a number of exposure times that are included in the plurality of frame images as described by Tsuzuki in the system and method for processing video images of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate a number of frame buffers that are included in the image processing device is equal to a number of exposure times that are included in the plurality of frame images, since it provides a mechanism to continuously capture and write images of different exposures of the same corresponding regions. (Id. at c.6, ll.5-10).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Tsutsumi (U.S. Patent No. 9,025,038) (“Tsutsumi”) as applied to claims 10 and 12 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”).
With respect to the limitations of claim 11, and
further comprising: a third frame buffer configured to receive a fourth frame image of the plurality of frame images between the second frame image and the third frame image, the fourth frame image having a third exposure time different from the first exposure time and the second exposure time, wherein the HDR processor is configured to generate the first HDR image by combining the first frame image, the second frame image and the fourth frame image, and generate the second HDR image by combining the second frame image, the third frame image and the fourth frame image (claim 11).
The Examiner finds that Yamaguchi, Tsuzuki and Tsutsumi discloses the limitations, as previously set forth, except for specifically calling for the third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times and generating HDR images based upon frame number/exposure time combinations as set forth in the claim above.
However, providing a third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times and generating HDR images based upon frame number/exposure time combinations as set forth in the claim above would be obvious to one of ordinary skill in the art.
The Examiner finds that Yamaguchi, for example, teaches mixing three, instead of two, image frames while shifting one frame at a time with the method being applicable to image signals recorded with three or more different exposure times. (Yamaguchi at c.7, ll.16-19). Similarly, in the same field of endeavor, the Examiner finds that Shikata further teaches combining three or more successive frames of different exposure times to create respective HDR images. (Shikata at ¶ 0115). Moreover, the Examiner finds that Tsuzuki teaches having a frame buffer dedicated to each respective exposure setting. (Tsuzuki at c.5, l.47 – c.6, l.13; see Figures 1, 2).
Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate a third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times set forth in the claims above from the teaching of Tsuzuki.
A person of ordinary skill in the art would be motivated to incorporate a third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times set forth in the claims above, since it provides a mechanism to continuously capture and write images of different exposures of the same corresponding regions. (Id. at c.6, ll.5-10)
Similarly, it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above from the teaching of Yamaguchi and Shikata.
A person of ordinary skill in the art would be motivated to incorporate c the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, since it provides a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Yamaguchi and Shikata to the system and method for processing a video image of Yamaguchi and Tsutsumi would have yielded predicable results and resulted in an improved system, namely, providing the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, in Yamaguchi and Tsutsumi to provide a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Claims 1, 3, 5, 13 and 14 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”).
With respect to the limitations of claims 1 and 13, and
[1] [a] method of image processing a video image including a plurality of frame images, the method comprising:
In this regard, the Examiner finds that Yamaguchi discloses an image processing system and method for processing a video image including a plurality of frame images. (Yamaguchi at Figures 1, 2; see mixer 6/frame memory 7 combination).
generating a first high dynamic range (HDR) image based on a first frame image and a second frame image of the plurality of frame images by (i) determining an amount of movement between the first frame image and the second frame image, (ii) shifting the first frame image or an object therein based on the amount of movement to generate a shifted first frame image, and (iii) combining the shifted first frame image and the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images, the first frame image having a first exposure time, the second frame image having a second exposure time different from the first exposure time; and
generating a second HDR image based on the second frame/reference image and a third frame image of the plurality of frame images by (i) determining an amount of movement between the second frame image and the third frame image, (ii) shifting the second frame image or an object therein based on the amount of movement to generate a shifted second frame image, and (iii) combining the shifted second frame image and the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images, the third frame image having the first exposure time
In this regard, the Examiner finds that Yamaguchi discloses the system and method generating: (1) a first high dynamic range (HDR) image based on a first frame image (Yamaguchi at Figure 2; -see the “nth frame”) and a second frame image (id.; see the (n+1)th frame) of the plurality of frame images by (iii) combining the first frame image and second frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]); and (2) a second HDR image based on the second frame image and a third frame image (id. at Figure 2; see the (n+2)th frame) of the plurality of frame image by (iii) combining the second frame image and third frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]). The Examiner finds that: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id.)
Yamaguchi discloses the limitations, as previously set forth, except for specifically calling for generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image.
However, generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image would be obvious to one of ordinary skill in the art.
The Examiner find that MGTKeiichii teaches, for example, an image capturing apparatus and method that captures images at various exposure times (MGTKeiichii at ¶¶ 0004-0007; 0008-0015; 0031-0033; 0034-0036; 0039-0052; see Figures 1-6) in which, before images of different exposures time are combined to form an image, the generating further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the second/third frame image or an object therein based on the amount of movement to generate a shifted second/third frame image, and (iii) combining the shifted second/third frame image and the first/second frame image. (Id.)
The Examiner finds that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the second/third frame image or an object therein based on the amount of movement to generate a shifted second/third frame image, and (iii) combining the shifted second/third frame image and the first/second frame image as described in MGTKeiichii in the system and method for processing a video image of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the second/third frame image or an object therein based on the amount of movement to generate a shifted second/third frame image, and (iii) combining the shifted second/third frame image and the first/second frame image, since it provides a mechanism to compensate for camera shake or motion of a subject between frames. (Id. at ¶¶ 0015).
The Examiner finds that MGTKeiichii teaches not shifting the reference frame (i.e., the first/second frame) , but instead shifting the input frame (i.e., the second/third frame) to compensate for motion. (MGTKeiichii at ¶¶ 0004-0007; 0008-0015; 0031-0033; 0034-0036; 0039-0052; see Figures 1-6). However, the Examiner finds that it would be obvious to shift the reference frame (i.e., the first/second frame) instead of shifting the input frame (i.e., the second/third frame) to compensate for motion.
From this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of providing a system and method for processing video images that utilizes motion detection between images of different exposures, as described in MGTKeiichii, in order to compensate for camera shake or motion of a subject between frames.
(2) A finding that MGTKeiichii provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images that utilizes motion detection between images of different exposures by either: 1) motion compensation on reference frame; or 2) motion compensation on input frame.
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since MGTKeiichii explicitly teaches providing a system and method for processing video images that utilizes motion detection between images of different exposures by either using: 1) motion compensation on reference frame; or 2) motion compensation on input frame, MGTKeiichii teaches that one of ordinary skill in the art could have pursued the known potential solution (i.e., using motion compensation on reference frame instead of the input frame) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., using motion compensation on reference frame instead of the input frame) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397. If any of these findings cannot be made, then this rationale cannot be used to support a conclusion that the claim would have been obvious to one of ordinary skill in the art.
With respect to the limitations of claim 3, and
[3] further comprising: generating a third HDR image based on the third frame image and a fourth frame image of the plurality of frame images, the fourth frame image being subsequent in time to the third frame image in the plurality of frame images, the fourth frame image having the second exposure time.
In this regard, the Examiner finds that Yamaguchi discloses the system and method generating a third HDR image based on the third frame image and a fourth frame image (Yamaguchi at Figure 2; see (n+3)th frame is a long-exposure image) of the plurality of frame images, the fourth frame image being subsequent in time to the third frame image in the plurality of frame images, the fourth frame image having the second exposure time (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the n+2th frame and the (n+3)th frame to create a third HDR image). The Examiner finds that: (1) the (n+2)th frame and the (n+3)th frame shown in Figure 2 correspond to “a third frame image” and “fourth frame image,” respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id.)
With respect to the limitations of claim 5, and
[5] wherein the determining the amount of movement includes: extracting a global movement between the first frame image and the second frame image.
In this regard, the Examiner finds that MGTKeiichii teaches the amount of movement being a global movement between first and second frame images. (MGTKeiichii at ¶¶ 0004-0007; 0008-0015; 0031-0033; 0034-0036; 0039-0052).
Thus, similarly, as set forth above, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determining the amount of movement to include extracting a global movement between the first frame image and the second frame image as described by MGTKeiichii in the system and method for processing a video image of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate the determining the amount of movement to include extracting a global movement between the first frame image and the second frame image, since it provides a mechanism to compensate for camera shake or motion of a subject between frames. (Id. at ¶¶ 0015).
With respect to the limitations of claim 14, and
[14] wherein the reference image is one of the second frame image or the first HDR image.
In this regard, the Examiner finds that Yamaguchi discloses the system and method generating: (1) a first high dynamic range (HDR) image based on a first frame image (Yamaguchi at Figure 2; -see the “nth frame”) and a second frame image (id.; see the (n+1)th frame) of the plurality of frame images. (Yamaguchi at c.9, ll.28-42; see Figure 2).
Claims 2, 4, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”).
With respect to the limitations of claims 2, 4, 16 and 18, and
[2, 16] wherein a number of the plurality of frame images received before the image processing is equal to a number of a plurality of HDR images obtained after the image processing, the first HDR image and the second HDR image being included in the plurality of HDR images (claims 2 and 16); and
[4, 18] further comprising: outputting a start frame image as a start HDR image without performing the image processing on the start frame image, the start frame image being at a head of the plurality of frame images (claims 4 and 18).
In this regard, the Examiner finds that Yamaguchi discloses the first HDR image and the second HDR image being included in the plurality of HDR images. (Yamaguchi at c.9, ll.37-42; see Figure 2). The Examiner finds that the given the fact that two input frames are required to generate a HDR image, the total number of HDR images will be one less than the total number of input frames. While Yamaguchi discloses the limitations, as set forth above, the Examiner finds that Yamaguchi and MGTKeiichii is silent on how to treat the first frame.
However, creating an HDR image from either a sequence of images (i.e., bracketing) or from a single image is known in the art. The Examiner finds that Herman, for example, teaches that an HDR image can be created from either a sequence of images (i.e., bracketing) or from a single image. (Herman at c.5, l.67 – c.6, l.6).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate creating an HDR image from a single first frame image as described by Herman in the system and method for processing video images of Yamaguchi and MGTKeiichii.
A person of ordinary skill in the art would be motivated to incorporate creating an HDR image from a single first frame image, since it provides a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Herman to the system and method for processing a video image of Yamaguchi and MGTKeiichii would have yielded predicable results and resulted in an improved system, namely, providing creating an HDR image from a single first frame image, in Yamaguchi and MGTKeiichii to provide a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Furthermore, and from this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of utilizing a system and method for processing video images, as described in Herman, in order to provide a mechanism to easily create HDR images.
(2) A finding that Herman provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images to create an HDR image by either (i.e., 1) bracketing a plurality of images; or 2) utilizing a single image).
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since Herman explicitly teaches the utilization of a system and method for processing video images to create an HDR image, with either using the bracketing a plurality of images or a single image to create an HDR image, Herman teaches that one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397.
Claims 6 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Thumpudi (U.S. Publication No. 2018/0367734)(“Thumpudi”).
With respect to the limitations of claims 6 and 20, and
[6] wherein the determining the amount of movement includes extracting a local movement for the object between the first frame image and the second frame image, the object being included in both the first frame image and the second frame image (claim 6); and
[20] wherein the amount of movement is a local movement for the object, and the shifting includes shifting the object in the reference image based on the local movement (claim 20).
In this regard, the Examiner finds that MGTKeiichii teaches the amount of movement being a global movement between first and second frame images. (MGTKeiichii at ¶¶ 0004-0007; 0008-0015; 0031-0033; 0034-0036; 0039-0052).
The Examiner finds that Yamaguchi and MGTKeiichii discloses the limitations, as previously set forth, except for specifically calling for the determining the amount of movement to include extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement.
However, a determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement is known in the art. The Examiner finds that Thumpudi, for example, teaches a system and method for detecting an amount of movement between a series of frames by utilizing global movement detection and/or local movement detection of an object. (Thumpudi at Abstract; ¶¶ 0005-0007; 0014-0015; 0019; 0023-0027; 0042-0043; 0047-0048; claim 1).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate the determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement as described by Thumpudi in the system and method for processing video images of Yamaguchi and MGTKeiichii.
A person of ordinary skill in the art would be motivated to incorporate the determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement, since it provides a mechanism to process less information to detect motion. (Id. at ¶¶ 0027; 0043).
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Thumpudi to the system and method for processing a video image of Yamaguchi and MGTKeiichii would have yielded predicable results and resulted in an improved system, namely, providing the determining the amount of movement including extracting a local movement for the object between the first frame image and the second frame image with the object being included in both the first frame image and the second frame image; and the amount of movement being a local movement for the object, and the shifting including shifting the object in the reference image based on the local movement, in Yamaguchi and MGTKeiichii to provide a mechanism to process less information to detect motion.
Furthermore, and from this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of utilizing a system and method for processing video images, as described in Thumpudi, in order to provide a mechanism to process less information to detect motion.
(2) A finding that Thumpudi provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images to detect motion in an image by either ((i.e., 1) extracting global movement; or 2) extracting local movement).
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since Thumpudi explicitly teaches the utilization of a system and method for processing video images to create an HDR image, with either using the extracting global movement or extracting local movement to detect motion, Thumpudi teaches that one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing extracting local movement to detect motion) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing extracting local movement to detect motion) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397.
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 1, 3, 5, 13 and 14 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”).
With respect to the limitations of claims 7 and 8 and
[7] wherein the generating the first HDR image generates the first HDR image based on the first frame image, the second frame image and a fourth frame image of the plurality of frame images between the second frame image and the third frame image, the fourth frame image having a third exposure time different from the first exposure time and the second exposure time, and the generating the second HDR image generates the second HDR image based on the second frame image, the third frame image, and the fourth frame image (claim 7); and
[8] further comprising: generating a third HDR image based on the third frame image, the fourth frame image and a fifth frame image of the plurality of frame images, the fifth frame image being subsequent in time to the third frame image in the plurality of frame images, the fifth frame image having the second exposure time (claim 8); and
The Examiner finds that Yamaguchi and MGTKeiichii discloses the limitations, as previously set forth, except for specifically calling for the frame number/exposure time combinations as set forth above to generate respective HDR images that correlate to the combinations.
However, generating HDR images based upon frame number/exposure time combinations as set forth in the claims above is known in the art. The Examiner finds that Yamaguchi, for example, teaches mixing three, instead of two, image frames while shifting one frame at a time with the method being applicable to image signals recorded with three or more different exposure times. (Yamaguchi at c.7, ll.16-19). Similarly, in the same field of endeavor, the Examiner finds that Shikata further teaches combining three or more successive frames of different exposure times to create respective HDR images. (Shikata at ¶ 0115).
Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above from the teaching of Yamaguchi and Shikata.
A person of ordinary skill in the art would be motivated to incorporate c the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, since it provides a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Yamaguchi and Shikata to the system and method for processing a video image of Yamaguchi and MGTKeiichii would have yielded predicable results and resulted in an improved system, namely, providing the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, in Yamaguchi and MGTKeiichii to provide a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) and Shikata (U.S. Publication No. 2016/0381302)(“Shikata”) as applied to claims 7 and 8 above, and in further view of Herman et al. (U.S. Patent No. 10,193,627) (“Herman”).
With respect to the limitations of claim 9, and
[9] further comprising: outputting a start frame image as a start HDR image without performing the image processing, the start frame image being at a head of the plurality of frame images.
In this regard, the Examiner finds that Yamaguchi discloses the first HDR image and the second HDR image being included in the plurality of HDR images. (Yamaguchi at c.9, ll.37-42; see Figure 2). The Examiner finds that the given the fact that two input frames are required to generate a HDR image, the total number of HDR images will be one less than the total number of input frames. While Yamaguchi discloses the limitations, as set forth above, the Examiner finds that Yamaguchi, MGTKeiichii and Shikata is silent on how to treat the first frame.
However, creating an HDR image from either a sequence of images (i.e., bracketing) or from a single image is known in the art. The Examiner finds that Herman, for example, teaches that an HDR image can be created from either a sequence of images (i.e., bracketing) or from a single image. (Herman at c.5, l.67 – c.6, l.6).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate creating an HDR image from a single first frame image as described by Herman in the system and method for processing video images of Yamaguchi, MGTKeiichii and Shikata.
A person of ordinary skill in the art would be motivated to incorporate creating an HDR image from a single first frame image, since it provides a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Herman to the system and method for processing a video image of Yamaguchi, MGTKeiichii and Shikata would have yielded predicable results and resulted in an improved system, namely, providing creating an HDR image from a single first frame image, in Yamaguchi, MGTKeiichii and Shikata to provide a mechanism to apply a process of creating an HDR image when the bracketing process is not available.
Furthermore, and from this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of utilizing a system and method for processing video images, as described in Herman, in order to provide a mechanism to easily create HDR images.
(2) A finding that Herman provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images to create an HDR image by either ((i.e., 1) bracketing a plurality of images; or 2) utilizing a single image).
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since Herman explicitly teaches the utilization of a system and method for processing video images to create an HDR image, with either using the bracketing a plurality of images or a single image to create an HDR image, Herman teaches that one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., utilizing a single image to create an HDR image) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397.
Claims 10 and 12 are rejected under 35 U.S.C. 103 as obvious over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”).
With respect to the limitations of claim 10, and
[10] [a]n image processing device configured to perform image processing on a video image including a plurality of frame images, the plurality of frame images including a first frame image, a second frame image and a third frame image, the image processing device comprising:
In this regard, the Examiner finds that Yamaguchi discloses an image processing system and method for processing a video image including a plurality of frame images. (Yamaguchi at Figures 1, 2; see mixer 6/frame memory 7 combination). The Examiner finds that Yamaguchi discloses: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively. (Id. at c.9, ll.28-42; and see Figure 2).
a first frame buffer configured to:
receive the first frame image, the first frame image having a first exposure time, and
receive the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images, the third frame image having the first exposure time
a second frame buffer configured to:
receive the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images, the second frame image having a second exposure time different from the first exposure time; and
In this regard, the Examiner finds that Yamaguchi discloses the image processing system comprising a memory unit 802 that is configured to receive the first frame image (Yamaguchi at Figure 2; -see the “nth frame”), the first frame image having a first exposure time, and receive the third frame image (id. at the (n+2)th frame), the third frame image being subsequent in time to the second frame image (id. at the (n+1)th frame) in the plurality of frame images, the third frame image having the first exposure time. (Id. at c.9, ll.28-42; and see Figure 2). In addition, the Examiner finds that the memory unit 802 being further configured to receive the second frame image (id. at the (n+1)th frame), the second frame image being subsequent in time to the first frame image (id. at the “nth frame” in the plurality of frame images. (Id.) The Examiner finds that: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id.)
Yamaguchi discloses the limitations, as previously set forth, except for specifically calling for the memory to comprise a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image.
However, a memory comprising a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image is known in the art. The Examiner finds that Tsuzuki, for example, teaches an image processing apparatus and method comprising a frame memory 20/20A comprising a short term exposure image buffer and a long exposure image buffer with the short term exposure image buffer configured to receive the first and third frame images and a long exposure image buffer configured to receive the second frame image. (Tsuzuki at c.5, l.47 – c.6, l.13; see Figures 1, 2).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate a memory comprising a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image as described by Tsuzuki in the system and method for processing video images of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate a memory comprising a first and second frame buffer with the first frame buffer configured to receive the first and third frame images and a second frame buffer configured to receive the second frame image, since it provides a mechanism to continuously capture and write images of different exposures of the same corresponding regions. (Id. at c.6, ll.5-10).
a high dynamic range (HDR) processor configured to:
generating a first HDR image based on the first frame image and the second frame image of the plurality of frame images by (i) determining an amount of movement between the first frame image and the second frame image, (ii) shifting the first frame image or an object therein based on the amount of movement to generate a shifted first frame image, and (iii) combining the shifted first frame image and the second frame image, the second frame image being subsequent in time to the first frame image in the plurality of frame images, the first frame image having a first exposure time, the second frame image having a second exposure time different from the first exposure time; and
generating a second HDR image based on the second frame image and the third frame image of the plurality of frame images by (i) determining an amount of movement between the second frame image and the third frame image, (ii) shifting the second frame image or an object therein based on the amount of movement to generate a shifted second frame image, and (iii) combining the shifted second frame image and the third frame image, the third frame image being subsequent in time to the second frame image in the plurality of frame images, the third frame image having the first exposure time
In this regard, the Examiner finds that Yamaguchi discloses the system and method comprising a CPU having memory and programs thereon (Yamaguchi at c.21, ll.45-58) to perform the operations of generating: (1) a first high dynamic range (HDR) image based on a first frame image (Yamaguchi at Figure 2; -see the “nth frame”) and a second frame image (id.; see the (n+1)th frame) of the plurality of frame images by (iii) combining the first frame image and second frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]); and (2) a second HDR image based on the second frame image and a third frame image (id. at Figure 2; see the (n+2)th frame) of the plurality of frame image by (iii) combining the second frame image and third frames, respectively (id. at c.6, Equation 1; c.9, ll.28-42; and see Figure 2; [i.e. “mixes recorded image signals while shifting one frame at a time, mixing the nth frame and the (n+1)th frame, the (n+1)th frame and the (n+2)th frame ...”]). The Examiner finds that: (1) the nth frame, the (n+1)th frame, and the (n+2)th frame shown in Figure 2 correspond to “a first frame image”, “a second frame image” and “a third frame image”, respectively; and (2) the short exposure time and the long exposure time alternately repeat every frame with the first and third frames having the same exposure time; and the second and fourth having the same exposure time. (Id).
Yamaguchi discloses the limitations, as previously set forth, except for specifically calling for generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image.
However, generating the first/second HDR images by further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the first/second frame image or an object therein based on the amount of movement to generate a shifted first/second frame image, and (iii) combining the shifted first/second frame image and the second/third frame image would be obvious to one of ordinary skill in the art.
The Examiner find that MGTKeiichii teaches, for example, an image capturing apparatus and method that captures images at various exposure times (MGTKeiichii at ¶¶ 0004-0007; 0008-0015; 0031-0033; 0034-0036; 0039-0052; see Figures 1-6) in which, before images of different exposures time are combined to form an image, the generating further including (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the second/third frame image or an object therein based on the amount of movement to generate a shifted second/third frame image, and (iii) combining the shifted second/third frame image and the first/second frame image. (Id.)
The Examiner finds that it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the second/third frame image or an object therein based on the amount of movement to generate a shifted second/third frame image, and (iii) combining the shifted second/third frame image and the first/second frame image as described in MGTKeiichii in the system and method for processing a video image of Yamaguchi.
A person of ordinary skill in the art would be motivated to incorporate the generating the first/second HDR images to further include (i) determining an amount of movement between the first/second frame image and the second/third frame image, (ii) shifting the second/third frame image or an object therein based on the amount of movement to generate a shifted second/third frame image, and (iii) combining the shifted second/third frame image and the first/second frame image, since it provides a mechanism to compensate for camera shake or motion of a subject between frames. (Id. at ¶¶ 0015).
The Examiner finds that MGTKeiichii teaches not shifting the reference frame (i.e., the first/second frame) , but instead shifting the input frame (i.e., the second/third frame) to compensate for motion. (MGTKeiichii at ¶¶ 0004-0007; 0008-0015; 0031-0033; 0034-0036; 0039-0052; see Figures 1-6). However, the Examiner finds that it would be obvious to shift the reference frame (i.e., the first/second frame) instead of shifting the input frame (i.e., the second/third frame) to compensate for motion.
From this perspective, this combination of references satisfies at least rationale E identified by the Supreme Court in KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1395-97 (2007): “ ‘Obvious to try’ – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success.” The elements of the Graham factual inquiry for supporting a finding of obviousness based on this rationale are provided below:
(1) A finding that at the time of the invention, there had been a recognized problem or need of providing a system and method for processing video images that utilizes motion detection between images of different exposures, as described in MGTKeiichii, in order to compensate for camera shake or motion of a subject between frames.
(2) A finding that MGTKeiichii provides a finite number of identified, predictable potential solutions of providing a system and method for processing video images that utilizes motion detection between images of different exposures by either: 1) motion compensation on reference frame; or 2) motion compensation on input frame.
(3) A finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success. Here, since MGTKeiichii explicitly teaches providing a system and method for processing video images that utilizes motion detection between images of different exposures by either using: 1) motion compensation on reference frame; or 2) motion compensation on input frame, MGTKeiichii teaches that one of ordinary skill in the art could have pursued the known potential solution (i.e., using motion compensation on reference frame instead of the input frame) with a reasonable expectation of success (i.e., Obvious to try).
From this perspective, the Examiner asserts choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success is obvious to one of ordinary skill in the art. That is, as set forth above, one of ordinary skill in the art could have pursued the known potential solutions (i.e., using motion compensation on reference frame instead of the input frame) with a reasonable expectation of success (i.e., Obvious to try).
Thus, the rationale to support a conclusion that the claim would have been obvious is that “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely that product [was] not of innovation but of ordinary skill and common sense. In that instance the fact that a combination was obvious to try might show that it was obvious under § 103.” KSR, 550 U.S. at 421, 82 USPQ2d at 1397. If any of these findings cannot be made, then this rationale cannot be used to support a conclusion that the claim would have been obvious to one of ordinary skill in the art.
With respect to the limitations of claim 12, and
[12] wherein a number of frame buffers that are included in the image processing device is equal to a number of exposure times that are included in the plurality of frame images.
In this regard, the Examiner finds that Yamaguchi discloses he image processing system comprising a memory unit 802 that is configured to receive both short and long exposure times. (Yamaguchi. at c.9, ll.28-42; and see Figure 2).
In addition, the Examiner finds that Tsuzuki, for example, teaches an image processing apparatus and method comprising a number of frame buffers that are included in the image processing device being equal to a number of exposure times that are included in the plurality of frame images. (Tsuzuki at c.5, l.47 – c.6, l.13; see Figures 1, 2).
The Examiner finds that that it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate a number of frame buffers that are included in the image processing device is equal to a number of exposure times that are included in the plurality of frame images as described by Tsuzuki in the system and method for processing video images of Yamaguchi, Tsuzuki and MGTKeiichii.
A person of ordinary skill in the art would be motivated to incorporate a number of frame buffers that are included in the image processing device is equal to a number of exposure times that are included in the plurality of frame images, since it provides a mechanism to continuously capture and write images of different exposures of the same corresponding regions. (Id. at c.6, ll.5-10).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yamaguchi et al. (U.S. Patent No. 9,460,532) (“Yamaguchi”) in view of Tsuzuki (U.S. Patent No. 9,544,505) (Tsuzuki) and Keiichii et al. (Japanese Publication No. JP 2007-202050) (“Keiichii”) (JPO Machine Generate Translation for English equivalent (“MGTKeiichii”) as applied to claims 10 and 12 above, and in further view of Shikata (U.S. Publication No. 2016/0381302)(“Shikata”).
With respect to the limitations of claim 11, and
further comprising: a third frame buffer configured to receive a fourth frame image of the plurality of frame images between the second frame image and the third frame image, the fourth frame image having a third exposure time different from the first exposure time and the second exposure time, wherein the HDR processor is configured to generate the first HDR image by combining the first frame image, the second frame image and the fourth frame image, and generate the second HDR image by combining the second frame image, the third frame image and the fourth frame image.
The Examiner finds that Yamaguchi, Tsuzuki and MGTKeiichii discloses the limitations, as previously set forth, except for specifically calling for the third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times and generating HDR images based upon frame number/exposure time combinations as set forth in the claim above.
However, providing a third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times and generating HDR images based upon frame number/exposure time combinations as set forth in the claim above would be obvious to one of ordinary skill in the art.
The Examiner finds that Yamaguchi, for example, teaches mixing three, instead of two, image frames while shifting one frame at a time with the method being applicable to image signals recorded with three or more different exposure times. (Yamaguchi at c.7, ll.16-19). Similarly, in the same field of endeavor, the Examiner finds that Shikata further teaches combining three or more successive frames of different exposure times to create respective HDR images. (Shikata at ¶ 0115). Moreover, the Examiner finds that Tsuzuki teaches having a frame buffer dedicated to each respective exposure setting. (Tsuzuki at c.5, l.47 – c.6, l.13; see Figures 1, 2).
Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate a third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times set forth in the claims above from the teaching of Tsuzuki.
A person of ordinary skill in the art would be motivated to incorporate a third frame buffer to receive a fourth frame image with a different exposure time than the first and second exposure times set forth in the claims above, since it provides a mechanism to continuously capture and write images of different exposures of the same corresponding regions. (Id. at c.6, ll.5-10)
Similarly, it would have been obvious to one of ordinary skill in the art at the time of the invention was made to incorporate the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above from the teaching of Yamaguchi and Shikata.
A person of ordinary skill in the art would be motivated to incorporate c the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, since it provides a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Moreover, the Examiner asserts that applying a known technique to a known device ready for improvement would yield predictable results. That is, it would have been recognized by one of ordinary skill in the art that applying the known technique taught by Yamaguchi and Shikata to the system and method for processing a video image of Yamaguchi and MGTKeiichii would have yielded predicable results and resulted in an improved system, namely, providing the generating of HDR images based upon frame number/exposure time combinations as set forth in the claims above, in Yamaguchi and MGTKeiichii to provide a mechanism to utilize more image contour characteristics of the same image to generate a more detailed HDR image.
Conclusion
Applicant is respectfully reminded that any suggestions or examples of claim language provided by the Examiner are just that—suggestions or examples—and do not constitute a formal requirement mandated by the Examiner. To be especially clear, any suggestion or example provided in this Office Action (or in any future office action) does not constitute a formal requirement mandated by the Examiner.
Should Applicant decide to amend the claims, Applicant is also reminded that—like always—no new matter is allowed. The Examiner therefore leaves it up to Applicant to choose the precise claim language of the amendment in order to ensure that the amended language complies with 35 U.S.C. § 112 1st paragraph.
Independent of the requirements under 35 U.S.C. § 112 1st paragraph, Applicant is also respectfully reminded that when amending a particular claim, all claim terms must have clear support or antecedent basis in the specification. See 37 C.F.R. § 1.75(d)(1) and MPEP § 608.01(o). Should Applicant amend the claims such that the claim language no longer has clear support or antecedent basis in the specification, an objection to the specification may result. Therefore, in these situations where the amended claim language does not have clear support or antecedent basis in the specification and to prevent a subsequent ‘Objection to the Specification’ in the next office action, Applicant is encouraged to either (1) re-evaluate the amendment and change the claim language so the claims do have clear support or antecedent basis or, (2) amend the specification to ensure that the claim language does have clear support or antecedent basis. See again MPEP § 608.01(o) (¶3). Should Applicant choose to amend the specification, Applicant is reminded that—like always—no new matter in the specification is allowed. See 35 U.S.C. § 132(a). If Applicant has any questions on this matter, Applicant is encouraged to contact the Examiner via the telephone number listed below.
Applicant is reminded of the obligation to apprise the Office of any prior or concurrent proceedings in which the ‘887 Patent is or was involved, such as interferences or trials before the Patent Trial and Appeal Board, other reissues, reexaminations, or litigations and the results of such proceedings.
In accordance with MPEP § 1406, the Examiner has reviewed and considered the prior art cited or ‘of record’ in the original prosecution of the ‘887 Patent. Applicant is reminded that a listing of the information cited or ‘of record’ in the original prosecution of the ‘887 Patent need not be resubmitted in this reissue application unless Applicant desires the information to be printed on a patent issuing from this reissue application.
Applicant is further reminded of the continuing obligation under 37 C.F.R. §1.56 to timely apprise the Office of any information which is material to patentability of the claims under consideration in this reissue application.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEPHEN J RALIS whose telephone number is (571)272-6227. The examiner can normally be reached on Monday-Friday 8:30am-5:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hetul Patel can be reached on 571-272-4184. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Stephen J. Ralis/Primary Examiner, Art Unit 3992
Conferees:
/Luke S. Wassum/Primary Examiner, Art Unit 3992 /H.B.P/Hetul Patel
Supervisory Patent Examiner, Art Unit 3992
SJR
08/04/2026
Prosecution Insights