DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 6 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 6 recites change a display form of an area of the composite image which does not appear to be cited in the specification or drawings, the Examiner would like to request a citation or response as to where the act of changing the display form is disclosed, or to clarify/reword the act of changing the display form.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites change a display form of an area of the composite image wherein the display form is not specified to be a display medium (a screen or monitor), a display format (the divisions or splits of the screen, or the resolution or display ratio), or a display method (imaging or output techniques). Due to the inconclusive claim language, Examiner is requesting clarity as to how the display form is changed, or what the display form pertains to.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12 are rejected under 35 U.S.C. 103 as being unpatentable over Kaida Hironori (Pat. Pub. JP-2018067846-A, herein after “Hironori”) in view of Nakao Manabu (Pat. Pub. JP-2019169108-A, herein after “Manabu”).
In regard to claims 1, 10, 11, & 12, Hironori teaches [a]n image processing apparatus, comprising:
at least one memory storing instructions “The ROM 102 is a rewritable nonvolatile memory, and stores parameters necessary for the operation of each block in addition to the control program for each block of the imaging apparatus 100” (Hironori, Page 2); and
at least one processor in communication with the at least one memory and configured to execute the instructions “one or more processors in the computer of the system or apparatus read and execute the program” (Hironori, Page 12), wherein execution of the instructions causes the at least one memory and the at least one processor to:
divide each of a plurality of images captured continuously over time into similar areas;
generate a composite image from the plurality of images by synthesis processing “in this synthesis process, a composite image is generated by multiplying pixels having the same coordinates in the reference image and the synthesis target image by a gain corresponding to the weight of the luminance and then averaging” (Hironori, Page 5) where multiple images are taken for the synthesis process; and
control a display device to display the composite image “The imaging apparatus 100 according to the first embodiment can be operated in a processing mode in which pseudo long-second images sequentially generated from a plurality of short-second images obtained by short-time shooting continuously performed at short time intervals are sequentially updated and displayed” (Hironori, Page 3) where the display device is made to display the captured images,
wherein the synthesis processing for each of the areas divided by the dividing is ended based on an end instruction.
Hironori fails to explicitly teach divide each of a plurality of images captured continuously over time into similar areas; and
wherein the synthesis processing for each of the areas divided by the dividing is ended based on an end instruction.
Manabu teaches divide each of a plurality of images captured continuously over time into similar areas “a composite image generated in advance is displayed when the predicted direction of the divided image matches the direction instructed by the user” (Manabu, Page 8) where the images may be divided into sections and “[t]he predetermined timing in this case is, for example, a timing when a predetermined time has elapsed since the display of the displayed image is started, a timing when a user gives an instruction (an instruction without a direction instruction such as pressing the Enter key), or the like” (Manabu, Page 8) where the timing may be set by a person or predetermined; and
wherein the synthesis processing for each of the areas divided by the dividing is ended based on an end instruction “the display control unit 15 determines whether an instruction to end the image display has been received” (Manabu, Page 8) where various end instructions may be given to terminate the process.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of generating a composite image from a number of images by a synthesis process taught by Hironori with the method of dividing a display into multiple parts taught by Manabu to produce a split display capable of showing multiple composite images. The motivation to do so would be to maintain one area of an image while including parts from other images in the same display.
In regard to claim 10, claim 1 is substantially similar to claim 10, hence the rejection analysis for claim 1 is also applied to claim 10. Hironori in view of Manabu teach the additional limitations of [a]n imaging apparatus comprising:
an image sensor “The imaging unit 105 is an imaging element such as a CCD or a CMOS sensor” (Hironori, Page 2);
at least one memory storing instructions (Hironori, Page 2); and
at least one processor in communication with the at least one memory and configured to execute the instructions (Hironori, Page 12), wherein execution of the instructions causes the at least one memory and the at least one processor to:
divide each of a plurality of images captured continuously over time into similar areas (Manabu, Page 8);
generate a composite image from the plurality of images by synthesis processing (Hironori, Page 5); and
control a display device to display the composite image (Hironori, Page 3),
wherein the synthesis processing for each of the areas divided by the dividing is ended based on an end instruction (Manabu, Page 8).
In regard to claim 11, claim 1 is substantially similar to claim 11, hence the rejection analysis for claim 1 is also applied to claim 11. Hironori in view of Manabu teach the additional limitations of [a]n image processing method executed by an image processing apparatus “The image processing apparatus of the present embodiment can be applied to digital cameras, digital video cameras, various mobile terminals such as smartphones and tablet terminals having camera functions” (Hironori, Page 2), comprising:
dividing each of a plurality of images captured continuously over time into similar areas (Manabu, Page 8);
generating a composite image by combining the plurality of images by synthesis processing (Hironori, Page 5);
displaying the composite image (Hironori, Page 3),
wherein the synthesis processing is ended for each of the divided areas based on an end instruction (Manabu, Page 8).
In regard to claim 12, claim 1 is substantially similar to claim 12, hence the rejection analysis for claim 1 is also applied to claim 12. Hironori in view of Manabu teach the additional limitations of [a] non-transitory computer-readable storage medium which stores a program for causing a computer to execute an image processing method “The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a recording medium, and one or more processors in the computer of the system or apparatus read and execute the program” (Hironori, Page 12) comprising:
dividing each of a plurality of images captured continuously over time into similar areas (Manabu, Page 8);
generating a composite image by combining the plurality of images by synthesis processing (Hironori, Page 5);
displaying the composite image (Hironori, Page 3),
wherein the synthesis processing is ended for each of the divided areas based on an end instruction (Manabu, Page 8).
In regard to claim 2, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 1, wherein the dividing divides each of the plurality of images into two or more areas “the dividing unit 11 assigns a divided image ID for identifying the divided image to each of the divided images. Hereinafter, the divided image with the divided image ID i is referred to as “divided image i”. In FIG. 2, the number in the frame representing the divided image is the divided image ID of the divided image” (Manabu, Page 3) where the division may be an unspecified number and the division sizes can be predetermined, one of ordinary skill in the art may know that dividing an image would produce more than one image area.
In regard to claim 3, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 1, further comprising:
an instruction member configured to provide instructions “the display control unit 15 receives display direction information that is instructed by the user and indicates a direction in which a divided image to be displayed next exists” (Manabu, Page 4) where a user can be an instruction member that provides instructions since the term instruction member does not limit the scope of the claim,
wherein the instruction member is capable of selecting at least one of the divided areas “in step S26, the process waits until the display direction is instructed by the user. And if the display control part 15 receives display direction information, a process will transfer to step S27” (Manabu, Page 7) where the user can select a direction to divide the image.
In regard to claim 4, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 1, wherein, as the composite image, a first composite image is generated from a plurality of images captured until a first time point “The composite image 304 is a pseudo long second image synthesized based on the short second image 301 and the short second image 302” (Hironori, Page 6) where a composite image is generated from a set of short second images, and a second composite image is newly generated using images captured after the first time point “the image processing unit 107 compares the composite image 304 generated from the short second image 301 and the short-second image 302 and the short-second image 303 that is later in time series than the short-second images 301 and 302, by the comparatively bright composite processing 331. Thus, a composite image 305 is generated” (Hironori, Page 6-7) where the generated composite image is compared with an image captured after the time point of the previous images and a new composite image is generated.
PNG
media_image1.png
554
906
media_image1.png
Greyscale
Hironori, Fig. 3, depicting the first composite image (item 304) generated by a plurality of images (items 301 and 302) at a first time point, and second composite image (item 305) generated by image (item 303) captured after the time point.
In regard to claim 5, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 1, wherein execution of the instructions causes the at least one memory and the at least one processor to: perform control to change the display on the display device every time a new composite image is generated “The display unit 108 also displays a live view image captured by the imaging unit 105. Although details will be described later, in the present embodiment, a pseudo long-second image sequentially generated from a plurality of short-second images obtained by short-second shooting continuously performed at short time intervals is used as a display image, and is sequentially displayed on the display unit 108. Image display such as update display is also possible” (Hironori, Page 3) where an updated display is produced as the plurality of short-second images are obtained.
In regard to claim 6, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 5, wherein execution of the instructions causes the at least one memory and the at least one processor to change a display form of an area of the composite image in which synthesis processing has ended, on the display device “the synthesized image generated in the previous comparative bright synthesis process in time series order is used as the reference image, and the short second image of the next frame in time series order is used as the synthesis target image, so that comparative bright synthesis is performed. Such processing is sequentially repeated in time series. Therefore, for example, when a pseudo long-second image is sequentially generated from a plurality of short-second images” (Hironori, Page 6) where a the long-second image is sequentially generated as synthesis is performed. Sequentially generating the image is read as changing the display form since the display form is not specified to be any particular form.
In regard to claim 8, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 1, wherein the composite image is an image to which an effect of longer exposure than the plurality of images “the system control unit 101 sets shooting parameters such as the exposure time and shooting interval of the short second shooting” (Hironori, Page 3) where an exposure time can be set, which has been captured over time and used to create the composite image, is applied “The composite image 304 is a pseudo long second image synthesized based on the short second image 301 and the short second image 302. The arrow t in the figure of the composite image 304 is a combination of the short-second image 301 and the double-headed arrow t in the short-second image 302, and the automobile within the time equivalent to the long-second exposure in the pseudo-long-second image (304)” (Hironori, Page 6) where the composite image is a pseudo long second image based on the short second images. The short second images are captured over time and are used to create the composite image.
PNG
media_image1.png
554
906
media_image1.png
Greyscale
Hironori, Fig. 3. Depicting a plurality of short second images (items 301, 302, and 303) which are synthesized and used to generate the composite image (item 304) that depicts the imaged object at multiple stages.
In regard to claim 9, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 1, wherein the composite image is created by any one of addition synthesis, addition averaging, and comparative brightening “As another example of the synthesis process, for example, a synthesis process in which pixels having the same coordinates in the reference image and the synthesis target image are subjected to addition averaging with a weight corresponding to the luminance of the pixel having a higher luminance” (Hironori, Page 5) where addition synthesis is used, and “the synthesized image generated in the previous comparative bright synthesis process in time series order is used as the reference image, and the short second image of the next frame in time series order is used as the synthesis target image, so that comparative bright synthesis is performed” (Hironori, Page 6) where comparative brightening is performed.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Hironori in view of Manabu and Abe Mitsuo et al. (Pat. Pub. JP-2018046542-A, herein after “Mitsuo”).
In regard to claim 7, Hironori in view of Manabu teach [t]he image processing apparatus according to claim 6.
Hironori in view of Manabu fail to explicitly teach wherein execution of the instructions causes the at least one memory and the at least one processor to highlight the area of the composite image in which the synthesis processing has ended.
Teaches wherein execution of the instructions causes the at least one memory and the at least one processor to highlight the area of the composite image in which the synthesis processing has ended “When two areas are designated by the user on the range designation screen 40, a highlight display 51 is made for the area corresponding to the depth synthesis range” (Mitsuo, Page 7) and “according to the digital camera 100, when two image areas are designated by the user, the image area to be synthesized for the depth synthesis, that is, the range of the subject to be focused is highlighted” (Mitsuo, Page 10) where synthesized images may be highlighted based on range or user input. Various additional emphasis displays are taught on pages 7-9 such as a warning area highlight and depth combination range highlight.
PNG
media_image2.png
733
447
media_image2.png
Greyscale
Mitsuo, Figs. 12A and 12B, depicting an object in a preview image (item 30) with areas designated by a grid, areas that meet a parameter based on the synthesis are given a highlight display (item 51).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of generating a composite image from a number of images by a synthesis process and dividing a display into multiple parts taught by Hironori and Manabu with the use of a highlighting feature taught by Mitsuo to produce a split display capable of showing multiple composite images and highlighting the image upon a criteria. The motivation to do so would be to have a section of the image stand out for the user’s attention “Such highlighting 51 allows the user to easily recognize the range (depth synthesis range) that he / she wants to focus on” (Mitsuo, Page 7).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Watanabe Shinobu (Pat Pub. JP-2014175734-A) discloses composite image generation utilizing a synthesis process with an instruction to end.
Tatsuyuki Kamimura (Pat. Pub. CN-106851088-A) discloses image capture and generation of a composite image using image synthesis as well as an instruction to end.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAIDEN ALEXANDER USSERY whose telephone number is (571)272-1192. The examiner can normally be reached Monday - Friday* 7:30AM - 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.A.U./Examiner, Art Unit 2611
/TAMMY GODDARD/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights